Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve wide review handling #21

Merged
merged 40 commits into from
Oct 26, 2023
Merged

Improve wide review handling #21

Changes from 15 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
2efe568
Improve wide review handling
plehegar Jun 30, 2021
5b9729d
Update index.html
plehegar Jul 1, 2021
d68ccb8
Update index.html
plehegar Jul 1, 2021
827c3b3
Update index.html
plehegar Jul 1, 2021
8c67a8e
Update index.html
plehegar Jul 1, 2021
d6c1553
Update index.html
plehegar Jul 1, 2021
ccd4078
Update index.html
plehegar Jul 1, 2021
2bb4e97
Update index.html
plehegar Jul 1, 2021
01f2881
Update index.html
plehegar Jul 1, 2021
89ec59b
Update index.html
plehegar Jul 1, 2021
86e2b04
Update index.html
plehegar Jul 1, 2021
49f593a
Update index.html
plehegar Jul 1, 2021
11f603a
Update index.html
plehegar Jul 1, 2021
6597ca8
Update index.html
plehegar Jul 1, 2021
c00eb79
Contributions from @r12a
plehegar Jul 1, 2021
82039f3
Improve wide review handling
plehegar Jun 30, 2021
a7b4a9c
Update index.html
plehegar Jul 1, 2021
b697bd3
Update index.html
plehegar Jul 1, 2021
f79b45b
Update index.html
plehegar Jul 1, 2021
3ab2952
Update index.html
plehegar Jul 1, 2021
1d336f0
Update index.html
plehegar Jul 1, 2021
ebbcfd4
Update index.html
plehegar Jul 1, 2021
e09e4e8
Update index.html
plehegar Jul 1, 2021
904debb
Update index.html
plehegar Jul 1, 2021
c595865
Update index.html
plehegar Jul 1, 2021
1a9add0
Update index.html
plehegar Jul 1, 2021
59a7f93
Update index.html
plehegar Jul 1, 2021
298ccef
Update index.html
plehegar Jul 1, 2021
b47e64d
Update index.html
plehegar Jul 1, 2021
3a240e1
Contributions from @r12a
plehegar Jul 1, 2021
6d7909c
Rease with main branch
plehegar Jul 1, 2021
3148fee
Update index.html
plehegar Jul 1, 2021
bda0c4a
Update index.html
plehegar Jul 1, 2021
6b45764
Update index.html
plehegar Jul 1, 2021
9e444fc
Merge branch 'wide-review-42' of github.com:w3c/documentreview into w…
plehegar Jul 1, 2021
d2b3c29
The GitHub template should address wide review
plehegar Jul 1, 2021
79a724c
Update index.html
plehegar Jul 6, 2021
89e323b
You always request horizontal review
plehegar Jul 9, 2021
45a0238
Highlight the need to publish in /TR
plehegar Jul 9, 2021
851fb38
Merge branch 'gh-pages' into wide-review-42
plehegar Oct 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 89 additions & 33 deletions index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
<body>
<h1>How to do Wide Review</h1>
<p>Getting <i class="term">early</i> and <i class="term">wide</i> review of a document is very important, yet in practice it can be challenging. This document provides some best practices for getting document review; it does not define explicit mandatory steps.</p>
<p>This page is linked to from <a rel="nofollow" class="external text" href="https://www.w3.org/Guide/">The Guide</a>. See also the <a href="https://www.w3.org/2020/Process-20200915/#wide-review">Wide Review</a> section in the W3C Process document.</p>
<p>This page is linked to from <a rel="nofollow" class="external text" href="https://www.w3.org/Guide/">The Guide</a>. See also the <a href="https://www.w3.org/Consortium/Process/#wide-review">Wide Review</a> section in the W3C Process document.</p>
<p><strong>Feedback on this document is welcome, preferably by <a href="https://github.com/w3c/documentreview/issues">raising an issue</a> or a pull request.</strong></p>
<div id="toc">
<h2 class="notoc">Contents</h2>
Expand All @@ -21,26 +21,102 @@ <h2 class="notoc">Contents</h2>





<section id="when_should_review_be_requested">
<h2>When should wide review be requested?</h2>
<p>A document is available for review from the moment it is first <a
href='https://www.w3.org/Consortium/Process/#publishing'>published</a>.
Working Groups should <a
href='https://www.w3.org/Consortium/Process/#formally-addressed'>formally
address</a> any substantive review comment about a technical report in a
timely manner.</p>
<p>Wide review should or must be requested:</p>
<ul>
<li>After a <a href="https://www.w3.org/Consortium/Process/#first-wd">First
Public Working Draft</a> is published (for most documents).
<p>Working Groups are often reluctant to make substantive changes to a mature
document, so reviewers should get a chance to send substantive technical
reviews as early as possible. This is especially important for <a
href="#how_to_get_horizontal_review">horizontal reviews</a>.</p>
</li>
<li>Before a document gets advanced to <a
href="https://www.w3.org/Consortium/Process/#transition-cr">Candidate
Recommendation</a> or gets updated as a <a
michael-n-cooper marked this conversation as resolved.
Show resolved Hide resolved
href="https://www.w3.org/Consortium/Process/#update-requests">Candidate
Recommendation Snapshot</a>.
<p>For those, the <a href="https://www.w3.org/Consortium/Process/">W3C
Process</a> requires a Group to show that the specification has received wide
review.</p>
</li>
<li>When a document is both reasonably stable and still flexible enough to
allow changes where issues are identified.</li>
<li>When new features are added after a document has already received wide
review (perhaps requesting a review limited to the new features).</li>
</ul>
</section>

<section id="who_to_ask_for_review">
plehegar marked this conversation as resolved.
Show resolved Hide resolved
<h2>Who to ask for review?</h2>

<p>Much of this document focuses on how and when to conduct horizontal reviews, but they are only a subset of a full wide review, which must also include other stakeholders including Web developers, technology providers and implementers not active in the Working Group, external groups and standards organizations working on related areas, etc. Here is a list of suggestions:</p>
<h2>Who to ask for wide review?</h2>

<p>The objective is to ensure that the entire set of stakeholders of the Web
community, including the general public, have adequate notice of the progress
of the Working Group and are able to actually perform reviews of and provide
plehegar marked this conversation as resolved.
Show resolved Hide resolved
comments on the specification. When considering <a
href="https://www.w3.org/Guide/transitions">requests to advance the maturity
level of the document</a>, the Director will consider who has been explicitly
offered a reasonable opportunity to review the document.</p>

<p>You must publish an updated technical report, with the Status of the Document
indicating clearly that you're looking for <em>wide review</em>, before inviting
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of our publications are inviting review, that is why we publish them. Let's not add more junk to the SOTD.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except that the other groups need a clearer signal that wide review is expected before moving to CR. It's not junk, it gets picked up by our tools for the past few years.

review from other stakeholders. Our <a
href='https://github.com/w3c/transition-notifier/blob/main/notify.js'>tooling</a>
monitors publications and propagates notifications to <a
href="mailto:[email protected]">[email protected]</a>
automatically (for example, <a
href='https://lists.w3.org/Archives/Public/public-review-announce/2021Jun/0008.html'>Candidate
Recommendation Snapshot: Payment Request API (Call for Wide Review)</a>).</p>

<p>You should then reach out to:</p>
plehegar marked this conversation as resolved.
Show resolved Hide resolved

<ul>
<li> Groups listed in the WG's charter, especially those who manage dependencies.</li>
<li> Groups jointly responsible for a particular document (if any) (duh!).</li>
<li> All horizontal groups (listed below). If it appears that one of those is not relevant (and is not listed in your charter), talk to them informally to confirm that.</li>
<li> Other groups, at your discretion, even if not in the WG charter, including other W3C groups, external organizations and SSOs, implementers, application developers, etc.</li>
<li>The general public. Consider using blog posts, social media, or other ways of alerting the public to your document and requesting feedback.</li>
<li>the groups listed in the WG's charter, especially those who manage
dependencies.</li>
<li>the groups jointly responsible for a particular document (if any).</li>
<li>the <a href="#how_to_get_horizontal_review">horizontal groups</a>. If it
plehegar marked this conversation as resolved.
Show resolved Hide resolved
appears that one of those is not relevant (even if not listed in your charter),
talk to them informally to confirm that. <em>Note that sending an email to the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure informal contact is adequate here for demonstrating HR as per the process? Isn't a "paper" trail needed that shows that the WG told the HR group about the change and the HR group said they had no comments?

Copy link
Member Author

@plehegar plehegar Jul 1, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default, the paper trail is expected but the Director did use judgment in the past. For example, we don't really expect an Accessibility review on the WebAssembly Core specification. @swickr , wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If i18n feels that a review is needed, they will ask the WG to start the formal process. If they agree that it's not needed, then we'd like to avoid unnecessary paperwork or process steps (mainly because we want to preserve as much bandwidth as possible).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Process does not seem to allow for WG's not requesting review. I understand from the HR group's perspective that they don't want unnecessary work; on the other hand, from a WG's perspective, they want to have a formal statement of "we did what we were required to do".

I wonder: is it actually less work to have the informal conversation than to review the formal request and respond with a "go ahead, no input from us"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nigel, i think this process related suggestion doesn't belong in this PR. Could you raise an issue?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's easier for the Director to conclude that no review was needed by i18n when there is an actual record from i18n, than relying on out of band conversation and uncheckable assertions.

Yep; explicitly recorded assertions are much preferred; thanks.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 89e323b

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a nit: i wrote 'process' with a lowercase 'p', ie. referring to the general process, rather than the Process document. Probably doesn't change anything.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Process does not require sign-off from every HRG. It requires it to transition without the Director's permission, but the Director can use their judgement to approve a transition that skips certain HRG reviews (such as the unnecessary a11y review of WebAssembly mentioned earlier).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When filing the transition request without the full set of HRG reviews, groups should probably explain why they think a particular HRG's review is unnecessary, and the Director can accept when it seems obvious and push back if he feels it's inappropriate or borderline. I think that would be a reasonable substitute to asking the HRGs to do more paperwork on things that are clearly irrelevant to their interests.

public-review-announce list alone is not sufficient to trigger horizontal
reviews. You will still need to formally request review from the Horizontal
Groups, as described below</em></li>
<li>the general public, including Web developers, technology providers and
implementers, application developers, etc. Consider:
<ul>
<li>sending a dedicated announcement to <a
href="mailto:[email protected]">[email protected]</a>
if needed (in case the default notice sent to that list is not enough).</li>
<li>using blog posts, social media, or other ways of alerting the public to
your document and requesting feedback.</li>
</ul>
</li>

<li>other groups, at your discretion, even if not in the WG charter, such as
other W3C groups, external organizations and SSOs working on related areas,
etc.</li>

</ul>
<p>Use <a rel="nofollow" class="external text" href="mailto:[email protected]">[email protected]</a> for general announcements regarding wide and horizontal reviews. <em>However, note that sending an email to this list alone is not sufficient to trigger horizontal reviews. You will still need to formally request review from the Horizontal Groups, as described below.</em></p>
</section>

<p><strong>Tip:</strong> consider tracking your wide review progress using a
GitHub issue, such as <a href="https://github.com/w3c/sensors/issues/299">issue
#299 of the Sensors API</a>. You can then simply point the Director to the
issue.</p>


<p>The reviews provided by the <a
href="https://www.w3.org/Guide/process/charter.html#horizontal-review">horizontal
groups</a>, a subset of a full wide review, do receive special attention and
much of the rest of this document focuses on how and when to conduct horizontal
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Split the sentence after "and" here. :)

reviews.</p>

</section>

<section id="how_to_get_horizontal_review">
<h2>How to get horizontal review</h2>
Expand Down Expand Up @@ -92,8 +168,6 @@ <h2>How to get horizontal review</h2>
</details>
</dd>



<dt>Privacy</dt>
<dd>Write a "Privacy Considerations" section for your document, taking into account the <a rel="nofollow" class="external text" href="https://w3ctag.github.io/security-questionnaire/">Self-Review Questionnaire: Security and Privacy</a>, <a rel="nofollow" class="external text" href="https://w3c.github.io/fingerprinting-guidance/">Mitigating Browser Fingerprinting in Web Specifications</a>, and <a rel="nofollow" class="external text" href="https://tools.ietf.org/html/rfc6973">RFC6973</a> then <a rel="nofollow" class="external text" href="https://github.com/w3cping/privacy-request/issues/new/choose">request a review via GitHub</a> from the <a rel="nofollow" class="external text" href="https://www.w3.org/Privacy/">Privacy Interest Group</a>.
<details>
Expand Down Expand Up @@ -132,24 +206,6 @@ <h2>How to get horizontal review</h2>
<p>Recognize that horizontal review groups may be resource limited and may only be able to do one review or may have difficulty scheduling your review quickly. Give them as much time as you can, consistent with asking for review while it is still reasonable to change the technology to accommodate the issues they find.</p>
</section>





<section id="when_should_review_be_requested">
<h2>When should review be requested?</h2>
<ul>
<li>For most documents, after a <em>First Public Working Draft</em> is published</li>
<li> Process-2019 requires wide review before a document is published at CR (Candidate Recommendation)</li>
<li> When a document is both reasonably stable and still flexible enough to allow changes where issues are identified</li>
<li> When new features are added after a document has already gotten wide review (perhaps requesting a review limited to the new features)</li>
</ul>
</section>





<section id="working_with_horizontal_review_labels">
<h2><a href="#working_with_horizontal_review_labels">Working with Horizontal Review labels</a></h2>

Expand Down