From 93104fac85757fb573056032c834ac77bbbe7c09 Mon Sep 17 00:00:00 2001 From: Sam Sneddon Date: Wed, 9 Aug 2023 00:00:07 +0100 Subject: [PATCH 01/22] Add a proposal for the Interop 2024 process --- 2024/planning-process.md | 187 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 187 insertions(+) create mode 100644 2024/planning-process.md diff --git a/2024/planning-process.md b/2024/planning-process.md new file mode 100644 index 0000000..d771376 --- /dev/null +++ b/2024/planning-process.md @@ -0,0 +1,187 @@ +# Proposal for Interop 2024 Process + +## The Purpose of the Interop Project + +*Improve interoperability significantly for the benefit of web developers and users.* + +The goal of the Interop Project is to improve the web by making it easier to make websites and web apps that work in every browser or browser engine at the same time. + +This is done by increasing the amount of “interoperability” between browsers — when each browser engine has implemented the same technology the exact same way, as bug-free as possible. + +Today’s browsers have made a commitment to implement web technology according to a shared web standard, created in organizations such as the W3C or WHATWG, where technologies such as CSS and HTML are officially defined. + +There is a seemingly infinite amount of work that browser engineering teams could be focused on. The Interop Project provides incentives to focus on the specific and practical work that will help web developers most in the coming year. + +## Scope + +The Interop Project is a collaboration between organizations that implement web technology in browser engines. It’s a shared agreement to prioritize work on certain web technologies to improve interoperability. And to keep track of that work by using automated tests to score how much progress each participating browser has made reaching the shared goals. + +There are many fantastic ideas for what the web could become. But the Interop Project is not the place to begin making those dreams a reality. This is a place to focus on interoperable implementations of ideas that have already been defined in detail in web standards. And it’s a place to focus on interoperability through the use of automated tests to evaluate whether or not a particular browser matches what the web standard says it should be doing. In order for this to be possible, the Interop Project only focuses on specific kinds of web technology. + +Everything chosen for the next year’s project must be defined in a sufficiently mature standards-track web specification. If there is no web standard, then there is no definition of what a particular technology “should” be doing. The Interop Project is not an organization for defining web standards. That work is important, and must happen in the appropriate bodies, not here. + +Everything chosen for the next year’s project must be testable using automated tests on existing testing platforms. A proposal cannot be accepted if there are no automated tests to evaluate conformance to the web standard; if it’s not possible to test the technology being proposed; or if the tests would need to run in environments that are not yet available on the infrastructure for Web Platform Tests (for example, in 2022 and 2023, testing on mobile operating systems was not possible). + +If there is a particular blocker making it hard to test a certain feature, that could make for a good Investigation Project proposal. And ideas about how the WPT/Interop Project testing infrastructure can be improved for the future can be proposed as Investigation Projects, to be considered as work for the Interop Project to take on in the next year. + +### **Requirements for Focus Area proposals** + +* This feature has a mature-enough web standard from an established standards development organization such as IETF, Khronos Group, TC39, WHATWG, or the W3C. +* This feature has high quality tests in WPT or Test262. + +### **Guidance for Prioritizing Focus Area proposals** + +We should prioritize areas where web developers have had problems using them because of differences between browsers, or because they aren’t implemented in all browsers. There are many sources for such information: + +* Perhaps there are issues filed about such problems in the Chromium, Gecko, or WebKit bug trackers. +* Perhaps there are many complaints about this feature in public discussions on sites like Stack Overflow. +* Perhaps this feature ranks in high-demand or as a pain-point consistently in polls and surveys of web developers. +* Perhaps there are documented examples of sites or libraries working around the fact that it is missing or not interoperable. + +When assessing the prioritization of existing web technology, it can be helpful to assess how often the technology is being used currently. (Such thinking should also consider important technology might not be in popular use if a lack of interoperability is a painful barrier.) + +* The feature has high usage via Chrome use counters. +* The feature has high usage via HTTP Archive. +* The feature has a high number of mentions on developer resources like Stack Overflow. + +There are also guiding values that should be taken into consideration, beyond pure demand: + +* This feature has a positive impact on accessibility. +* This feature has a positive impact on internationalization. +* This feature has a positive impact on privacy. +* This feature has a positive impact on security. + + + +## Process + +### Timeline — 10 weeks total + +1. Call for Proposal Period — three weeks (Sep 14 – Oct 5) +2. Continued discussion and refinement of proposals — one week (Oct 5 – Oct 12) +3. Elimination Round 1: Disqualification — one week (Oct 12 – Oct 19) +4. Elimination Round 2: Reduction — two weeks (Oct 19 – Nov 2) +5. Elimination Round 3: Prioritization — three weeks (Nov 2 – Nov 30, exc. North America Thanksgiving) + + + +### Call for Proposal Period + +An open call for proposals will begin on a specific date, and run for three weeks. + +1. Anyone can submit a proposal. +2. Proposals should be for either: + 1. Focus Area, or + 2. Investigation Project +3. Related proposals should be filed separately. For example, all the ideas for “Web Compat” should be filed as a set of separate issues, not one issue. This makes it easier to discuss and prioritize each separately. Bundles of proposals risk being rejected or deprioritized. +4. Focus Area proposals must: + 1. Name a specific web technology. + 1. This can be a web technology that has not yet been implemented in any or all browser engines. + 2. Or this can be a web technology that has already been implemented in all three engines, but those implementations have multiple test failures that reveal a lack of compliance to the web standard, causing a lack of interoperability. + 2. The feature must be defined in a sufficiently mature standards-track web specification from one the following organizations. Please provide link to the web standard. + 1. [IETF](https://www.ietf.org/) (Standard Track) + 2. [Khronos Group](https://www.khronos.org/) + 3. [TC39](https://tc39.es/) (Stage 2+) + 4. [W3C](https://www.w3.org/) (Recommendation Track) + 5. [WHATWG](https://whatwg.org/) + 3. Describe the benefit to web developers and the web itself + 4. Link to existing tests (if any) and/or describe plan to create needed tests +5. Investigation Project proposals must: + 1. Outline their scope and their goals, + 2. Not substantially be work better suited to a group within a standards development organization, + 3. Clearly provide present or future impact on cross-browser interoperability. + +As soon as each proposal is submitted as a GitHub issue, discussion of that proposal can begin. Questions can be asked. The proposal can be edited and refined. + +At the end of the three weeks, the deadline will end the period when brand new proposals can be submitted. + +A fourth week will allow for continued discussion and refinement. The deadline at the end of the fourth week ends the opportunity to significantly change the scope of the proposal. The proposal will be evaluated on its state at that time. + + +## Focus Area Selection + +### Elimination Round 1 : Disqualification + +After the Call of Proposals Period has ended, the organizing committee will finalize an official list of all Focus Area proposals. + +Any Focus Area proposal that does not meet the minimum publicly-stated selection criteria will be considered for preliminary elimination. For example: + +1. The proposal is overly vague or broad. +2. The proposed technology is already interoperable according to WPT results. +3. The proposed technology is not on a standards track, or its standard is not sufficiently mature. + +Decisions will be made at a specific meeting for discussion of preliminary eliminations. Any member organization can bring up a proposal they believe should be eliminated. Brief debate can be had. If **there is quick consensus**, that proposal will be eliminated. If there is not consensus and a longer debate is needed, the proposal will be not be eliminated and will instead move on to the next round. + +1. Setup *Disqualification meeting* +2. Each organization can propose items for disqualification. +3. Brief debate on each proposal: + 1. if there is quick consensus, disqualify the proposal + 2. otherwise, send the proposal to Round 2 +4. Publish the list of proposals pushed to Round 2. + +### Elimination Round 2: Reduction + +At this point, there is a clear list of Focus Area proposals under consideration. Each organization has two weeks to review the list of proposals and make Round 2 decisions. + +Each member organization chooses `N` number of proposals they would like to see move on to Round 3. There is no ranking or scoring at this point, a simple “yes, we want this proposal to stay in consideration”. Once all the “yeses” are gathered, any proposal that does not have the support of at least one organization is eliminated. + +For example, let’s imagine there are 112 focus area proposals submitted. Let’s set `N` to 10 — so each of the six organizations gets to choose 10 proposals. There will likely be significant overlap, but not complete. So imagine 32 proposals get chosen by at least one organization. (It will be no less than 10, and no more than 60.) This means the remaining 80 proposals are eliminated. + +If `N` is increased, then more proposals will make it into the next round. The goal is to find a balance so all of the proposals that are most likely to be considered high priority will make it into the next round. While eliminating as many as possible, to reduce the workload of the Interop 2024 organizations. There’s no need to extensively evaluate proposals of lower priority. + +### Elimination Round 3: Prioritization + +The official list of Focus Area proposals is updated to reflect the elimination of proposals not chosen by any organization. Each organization now has 20 days to review the list of remaining proposals and make their Round 3 decisions. + +In this round, each organization ranks the proposals in order of importance from most to least. And submits any vetos they may wish to exercise. + +Any proposal that gets a veto is immediately eliminated. + +Each proposal gets assigned points from each organization, according to their rank, and an ordered list of all remaining proposals is created. Where `X` is number of proposals that are ranked, and `Y` is the ranking a particular proposal gets from a certain organization, then that proposal gets `X-Y` points from that organization. Once all 6 rankings are in for that proposal, the total points from all six organizations is known. If that proposal is not eliminated by a veto, then it has a certain number of points, and can be listed in order among the other proposals. + +For example, if there are 20 remaining proposals, each organization will choose a number 1, number 2, down to number 20. Let’s say a proposal for flying cars gets the following ranks: 1, 5, 7, 10, 14, 20. That will give it the following points: 20, 15, 13, 10, 6, 1 = 65 points total. + +(If there appear to be quite a few proposals remaining in this round, the group can decide to only ask for each organization to submit their Top `X` number, while rating the rest zero. For example, if 50 proposals remain at this point, it likely makes sense to only submit a Top 25, and to not worry about ranking 26-50 in a specific order.) + +After Round 3, every remaining proposal can be listed in order, with the group’s highest priority at the top, and the lowest at the bottom. + +NOTE: in the past, “Exclude” was a way to express many different kinds of sentiments — including: does not meet the minimum qualifications, or this is low priority to us. We expect “veto” to be used much less than “exclude”, because there are other ways to express those sentiments. Vetoing is specifically for objecting to a particular proposal being included *at all,* no matter how highly ranked other organizations might find it. It’s for expressing objection to a particular web technology, perhaps to something an organization has already expressed an objection to in the web standards process or otherwise has an objection to implementing. + +## Interop 2023 Evaluation + +During the same period organizations are making Round 3 decisions, the organizing committee should evaluate the status of Interop 2023, and decide which Focus Areas will be retired, and which will be carried over. + +\[XXX] + +* * * + +# Decision point — what to do about Interop 2023? + +## There seem to be 3 options: + +### 1. Assume carryover. Retire some. Add how many ever that leaves space for. + +1. Start with the assumption that all Interop 2023 projects will be carried over — except for the ones that are “complete” + 1. Figure out a process for determining what should be retired. + 1. Perhaps by voting for each proposal one-by-one, to keep it or not (that’s what we did last year). + 2. Perhaps by score — everything above 95% Interop gets retired, or something similar. + 2. Decide how many total Focus Areas we want for Interop 2024. Perhaps 20? 15? + 3. That determines how many “open slots” we have. + 1. 26 - the number retired = the number carried over. + 2. If we are aiming for 20 areas in 2024, and 18 are carried over (retiring 6)... then we have space to add 2 new areas. + +### 2. Automatically ‘reapply’ all Focus Areas + +1. Carryover nothing. +2. Take all 26 Focus Areas from Interop 2023, and put them into the selection process. + +### 3. Make every Focus Area ‘reapply’ + +1. Carryover nothing. +2. Let people propose any focus area they believe is worthwhile still having included. + +* * * + +# Decision point — Investigation Project Selection + +Largely the same as Interop 2023? From 678df78be02f2c75296cf5d96ff71b0259cd87d9 Mon Sep 17 00:00:00 2001 From: Sam Sneddon Date: Fri, 18 Aug 2023 18:49:36 +0100 Subject: [PATCH 02/22] Move opening two sections of planning-process.md to new PR --- 2024/planning-process.md | 53 ---------------------------------------- 1 file changed, 53 deletions(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index d771376..bf70f4a 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -1,58 +1,5 @@ # Proposal for Interop 2024 Process -## The Purpose of the Interop Project - -*Improve interoperability significantly for the benefit of web developers and users.* - -The goal of the Interop Project is to improve the web by making it easier to make websites and web apps that work in every browser or browser engine at the same time. - -This is done by increasing the amount of “interoperability” between browsers — when each browser engine has implemented the same technology the exact same way, as bug-free as possible. - -Today’s browsers have made a commitment to implement web technology according to a shared web standard, created in organizations such as the W3C or WHATWG, where technologies such as CSS and HTML are officially defined. - -There is a seemingly infinite amount of work that browser engineering teams could be focused on. The Interop Project provides incentives to focus on the specific and practical work that will help web developers most in the coming year. - -## Scope - -The Interop Project is a collaboration between organizations that implement web technology in browser engines. It’s a shared agreement to prioritize work on certain web technologies to improve interoperability. And to keep track of that work by using automated tests to score how much progress each participating browser has made reaching the shared goals. - -There are many fantastic ideas for what the web could become. But the Interop Project is not the place to begin making those dreams a reality. This is a place to focus on interoperable implementations of ideas that have already been defined in detail in web standards. And it’s a place to focus on interoperability through the use of automated tests to evaluate whether or not a particular browser matches what the web standard says it should be doing. In order for this to be possible, the Interop Project only focuses on specific kinds of web technology. - -Everything chosen for the next year’s project must be defined in a sufficiently mature standards-track web specification. If there is no web standard, then there is no definition of what a particular technology “should” be doing. The Interop Project is not an organization for defining web standards. That work is important, and must happen in the appropriate bodies, not here. - -Everything chosen for the next year’s project must be testable using automated tests on existing testing platforms. A proposal cannot be accepted if there are no automated tests to evaluate conformance to the web standard; if it’s not possible to test the technology being proposed; or if the tests would need to run in environments that are not yet available on the infrastructure for Web Platform Tests (for example, in 2022 and 2023, testing on mobile operating systems was not possible). - -If there is a particular blocker making it hard to test a certain feature, that could make for a good Investigation Project proposal. And ideas about how the WPT/Interop Project testing infrastructure can be improved for the future can be proposed as Investigation Projects, to be considered as work for the Interop Project to take on in the next year. - -### **Requirements for Focus Area proposals** - -* This feature has a mature-enough web standard from an established standards development organization such as IETF, Khronos Group, TC39, WHATWG, or the W3C. -* This feature has high quality tests in WPT or Test262. - -### **Guidance for Prioritizing Focus Area proposals** - -We should prioritize areas where web developers have had problems using them because of differences between browsers, or because they aren’t implemented in all browsers. There are many sources for such information: - -* Perhaps there are issues filed about such problems in the Chromium, Gecko, or WebKit bug trackers. -* Perhaps there are many complaints about this feature in public discussions on sites like Stack Overflow. -* Perhaps this feature ranks in high-demand or as a pain-point consistently in polls and surveys of web developers. -* Perhaps there are documented examples of sites or libraries working around the fact that it is missing or not interoperable. - -When assessing the prioritization of existing web technology, it can be helpful to assess how often the technology is being used currently. (Such thinking should also consider important technology might not be in popular use if a lack of interoperability is a painful barrier.) - -* The feature has high usage via Chrome use counters. -* The feature has high usage via HTTP Archive. -* The feature has a high number of mentions on developer resources like Stack Overflow. - -There are also guiding values that should be taken into consideration, beyond pure demand: - -* This feature has a positive impact on accessibility. -* This feature has a positive impact on internationalization. -* This feature has a positive impact on privacy. -* This feature has a positive impact on security. - - - ## Process ### Timeline — 10 weeks total From 8071119495861d3cb2a7dc53c7a530e54f191720 Mon Sep 17 00:00:00 2001 From: Sam Sneddon Date: Wed, 6 Sep 2023 10:33:35 +0100 Subject: [PATCH 03/22] fixup! Add a proposal for the Interop 2024 process --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index bf70f4a..a8453cd 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -24,7 +24,7 @@ An open call for proposals will begin on a specific date, and run for three week 4. Focus Area proposals must: 1. Name a specific web technology. 1. This can be a web technology that has not yet been implemented in any or all browser engines. - 2. Or this can be a web technology that has already been implemented in all three engines, but those implementations have multiple test failures that reveal a lack of compliance to the web standard, causing a lack of interoperability. + 2. Or this can be a web technology that has already been implemented in all three engines, but those implementations have test failures that reveal a lack of compliance to the web standard, causing a lack of interoperability. 2. The feature must be defined in a sufficiently mature standards-track web specification from one the following organizations. Please provide link to the web standard. 1. [IETF](https://www.ietf.org/) (Standard Track) 2. [Khronos Group](https://www.khronos.org/) From e22cdc32cfa6b5e6a47bf815701a7c5140e785a3 Mon Sep 17 00:00:00 2001 From: Sam Sneddon Date: Wed, 6 Sep 2023 10:36:30 +0100 Subject: [PATCH 04/22] fixup! Add a proposal for the Interop 2024 process --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index a8453cd..a09b3f9 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -29,7 +29,7 @@ An open call for proposals will begin on a specific date, and run for three week 1. [IETF](https://www.ietf.org/) (Standard Track) 2. [Khronos Group](https://www.khronos.org/) 3. [TC39](https://tc39.es/) (Stage 2+) - 4. [W3C](https://www.w3.org/) (Recommendation Track) + 4. [W3C](https://www.w3.org/) ([Recommendation Track](https://www.w3.org/2023/Process-20230612/#rec-track)) 5. [WHATWG](https://whatwg.org/) 3. Describe the benefit to web developers and the web itself 4. Link to existing tests (if any) and/or describe plan to create needed tests From afe00cb51c51c6f3258a8fd31a38957d16d3d051 Mon Sep 17 00:00:00 2001 From: Sam Sneddon Date: Wed, 6 Sep 2023 10:37:12 +0100 Subject: [PATCH 05/22] fixup! Add a proposal for the Interop 2024 process --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index a09b3f9..e3edc1c 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -42,7 +42,7 @@ As soon as each proposal is submitted as a GitHub issue, discussion of that prop At the end of the three weeks, the deadline will end the period when brand new proposals can be submitted. -A fourth week will allow for continued discussion and refinement. The deadline at the end of the fourth week ends the opportunity to significantly change the scope of the proposal. The proposal will be evaluated on its state at that time. +A fourth week will allow for continued discussion and refinement of already-submitted proposals. The deadline at the end of the fourth week ends the opportunity to significantly change the scope of proposals. The proposals will be evaluated on their state at that time. ## Focus Area Selection From b2ae8b3732434e8e431fb4fc09c541bd5fcaad4c Mon Sep 17 00:00:00 2001 From: Sam Sneddon Date: Wed, 6 Sep 2023 10:38:38 +0100 Subject: [PATCH 06/22] fixup! Add a proposal for the Interop 2024 process --- 2024/planning-process.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index e3edc1c..628fc08 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -49,7 +49,7 @@ A fourth week will allow for continued discussion and refinement of already-subm ### Elimination Round 1 : Disqualification -After the Call of Proposals Period has ended, the organizing committee will finalize an official list of all Focus Area proposals. +After the Call of Proposals Period has ended, the Interop Team will finalize an official list of all Focus Area proposals. Any Focus Area proposal that does not meet the minimum publicly-stated selection criteria will be considered for preliminary elimination. For example: @@ -96,7 +96,7 @@ NOTE: in the past, “Exclude” was a way to express many different kinds of se ## Interop 2023 Evaluation -During the same period organizations are making Round 3 decisions, the organizing committee should evaluate the status of Interop 2023, and decide which Focus Areas will be retired, and which will be carried over. +During the same period organizations are making Round 3 decisions, the Interop Team should evaluate the status of Interop 2023, and decide which Focus Areas will be retired, and which will be carried over. \[XXX] From 8e9c82d43613e722834edf8166724d41193d8cd6 Mon Sep 17 00:00:00 2001 From: Sam Sneddon Date: Wed, 6 Sep 2023 11:37:06 +0100 Subject: [PATCH 07/22] fixup! Add a proposal for the Interop 2024 process --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index 628fc08..9c4bb7b 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -47,7 +47,7 @@ A fourth week will allow for continued discussion and refinement of already-subm ## Focus Area Selection -### Elimination Round 1 : Disqualification +### Elimination Round 1: Disqualification After the Call of Proposals Period has ended, the Interop Team will finalize an official list of all Focus Area proposals. From 1fb6dbb8e12664e09f8a1ffdd078b0e1dca60e64 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Philip=20J=C3=A4genstedt?= Date: Thu, 7 Sep 2023 18:19:42 +0200 Subject: [PATCH 08/22] Apply suggestions from code review --- 2024/planning-process.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index 9c4bb7b..b141981 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -2,7 +2,7 @@ ## Process -### Timeline — 10 weeks total +### Timeline 1. Call for Proposal Period — three weeks (Sep 14 – Oct 5) 2. Continued discussion and refinement of proposals — one week (Oct 5 – Oct 12) @@ -10,7 +10,7 @@ 4. Elimination Round 2: Reduction — two weeks (Oct 19 – Nov 2) 5. Elimination Round 3: Prioritization — three weeks (Nov 2 – Nov 30, exc. North America Thanksgiving) - +Interop 2024 and the set of accepted proposals will be announced in January 2024. ### Call for Proposal Period From d825b88a297af7a0e725869da957336ac5375433 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Thu, 28 Sep 2023 12:23:09 -0400 Subject: [PATCH 09/22] Update planning-process.md Setting `N` to be 40 for the purposes of round 2. --- 2024/planning-process.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index b141981..2630b9a 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -70,11 +70,9 @@ Decisions will be made at a specific meeting for discussion of preliminary elimi At this point, there is a clear list of Focus Area proposals under consideration. Each organization has two weeks to review the list of proposals and make Round 2 decisions. -Each member organization chooses `N` number of proposals they would like to see move on to Round 3. There is no ranking or scoring at this point, a simple “yes, we want this proposal to stay in consideration”. Once all the “yeses” are gathered, any proposal that does not have the support of at least one organization is eliminated. +Each member organization chooses 40 number of proposals they would like to see move on to Round 3. There is no ranking or scoring at this point, a simple “yes, we want this proposal to stay in consideration”. Once all the “yeses” are gathered, any proposal that does not have the support of at least one organization is eliminated. -For example, let’s imagine there are 112 focus area proposals submitted. Let’s set `N` to 10 — so each of the six organizations gets to choose 10 proposals. There will likely be significant overlap, but not complete. So imagine 32 proposals get chosen by at least one organization. (It will be no less than 10, and no more than 60.) This means the remaining 80 proposals are eliminated. - -If `N` is increased, then more proposals will make it into the next round. The goal is to find a balance so all of the proposals that are most likely to be considered high priority will make it into the next round. While eliminating as many as possible, to reduce the workload of the Interop 2024 organizations. There’s no need to extensively evaluate proposals of lower priority. +The goal is to find a balance so all of the proposals that are most likely to be considered high priority will make it into the next round. While eliminating as many as possible, to reduce the workload of the Interop 2024 organizations. There’s no need to extensively evaluate proposals of lower priority. ### Elimination Round 3: Prioritization From 6417e5c10228ae1069eb477ad44c803a069e1e54 Mon Sep 17 00:00:00 2001 From: Sam Sneddon Date: Wed, 4 Oct 2023 13:32:46 -0700 Subject: [PATCH 10/22] Update 2024/planning-process.md Co-authored-by: Dan Clark --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index 2630b9a..40d6be5 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -70,7 +70,7 @@ Decisions will be made at a specific meeting for discussion of preliminary elimi At this point, there is a clear list of Focus Area proposals under consideration. Each organization has two weeks to review the list of proposals and make Round 2 decisions. -Each member organization chooses 40 number of proposals they would like to see move on to Round 3. There is no ranking or scoring at this point, a simple “yes, we want this proposal to stay in consideration”. Once all the “yeses” are gathered, any proposal that does not have the support of at least one organization is eliminated. +Each member organization chooses 40 proposals they would like to see move on to Round 3. There is no ranking or scoring at this point, a simple “yes, we want this proposal to stay in consideration”. Once all the “yeses” are gathered, any proposal that does not have the support of at least one organization is eliminated. The goal is to find a balance so all of the proposals that are most likely to be considered high priority will make it into the next round. While eliminating as many as possible, to reduce the workload of the Interop 2024 organizations. There’s no need to extensively evaluate proposals of lower priority. From 70939802be67b4354a0436271cdf13b923bdcc1d Mon Sep 17 00:00:00 2001 From: Sam Sneddon Date: Wed, 4 Oct 2023 18:20:35 -0700 Subject: [PATCH 11/22] Note parts are confidential --- 2024/planning-process.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index 40d6be5..489a1e2 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -57,7 +57,7 @@ Any Focus Area proposal that does not meet the minimum publicly-stated selection 2. The proposed technology is already interoperable according to WPT results. 3. The proposed technology is not on a standards track, or its standard is not sufficiently mature. -Decisions will be made at a specific meeting for discussion of preliminary eliminations. Any member organization can bring up a proposal they believe should be eliminated. Brief debate can be had. If **there is quick consensus**, that proposal will be eliminated. If there is not consensus and a longer debate is needed, the proposal will be not be eliminated and will instead move on to the next round. +Decisions will be made at a specific meeting for discussion of preliminary eliminations. The minutes of this meeting are confidential to the team. Any member organization can bring up a proposal they believe should be eliminated. Brief debate can be had. If **there is quick consensus**, that proposal will be eliminated. If there is not consensus and a longer debate is needed, the proposal will be not be eliminated and will instead move on to the next round. 1. Setup *Disqualification meeting* 2. Each organization can propose items for disqualification. @@ -70,7 +70,7 @@ Decisions will be made at a specific meeting for discussion of preliminary elimi At this point, there is a clear list of Focus Area proposals under consideration. Each organization has two weeks to review the list of proposals and make Round 2 decisions. -Each member organization chooses 40 proposals they would like to see move on to Round 3. There is no ranking or scoring at this point, a simple “yes, we want this proposal to stay in consideration”. Once all the “yeses” are gathered, any proposal that does not have the support of at least one organization is eliminated. +Each member organization chooses 40 proposals they would like to see move on to Round 3; these selections are confidential to the team. There is no ranking or scoring at this point, a simple “yes, we want this proposal to stay in consideration”. Once all the “yeses” are gathered, any proposal that does not have the support of at least one organization is eliminated. The goal is to find a balance so all of the proposals that are most likely to be considered high priority will make it into the next round. While eliminating as many as possible, to reduce the workload of the Interop 2024 organizations. There’s no need to extensively evaluate proposals of lower priority. @@ -78,7 +78,7 @@ The goal is to find a balance so all of the proposals that are most likely to be The official list of Focus Area proposals is updated to reflect the elimination of proposals not chosen by any organization. Each organization now has 20 days to review the list of remaining proposals and make their Round 3 decisions. -In this round, each organization ranks the proposals in order of importance from most to least. And submits any vetos they may wish to exercise. +In this round, each organization ranks the proposals in order of importance from most to least. And submits any vetos they may wish to exercise. The entire contents of this round is confidential to the team. Any proposal that gets a veto is immediately eliminated. From 6a599d3dbfae78f0544eeb5b0c1f1b9571765b20 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Wed, 11 Oct 2023 15:53:39 -0400 Subject: [PATCH 12/22] Update 2024/planning-process.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Philip Jägenstedt --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index 489a1e2..3af6ba6 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -88,7 +88,7 @@ For example, if there are 20 remaining proposals, each organization will choose (If there appear to be quite a few proposals remaining in this round, the group can decide to only ask for each organization to submit their Top `X` number, while rating the rest zero. For example, if 50 proposals remain at this point, it likely makes sense to only submit a Top 25, and to not worry about ranking 26-50 in a specific order.) -After Round 3, every remaining proposal can be listed in order, with the group’s highest priority at the top, and the lowest at the bottom. +After Round 3, every remaining proposal can be listed in order, with the group’s highest priority at the top, and the lowest at the bottom. The team will decide by consensus which of the proposals will be included in Interop 2024 and how they will be grouped into focus areas and investigation efforts. NOTE: in the past, “Exclude” was a way to express many different kinds of sentiments — including: does not meet the minimum qualifications, or this is low priority to us. We expect “veto” to be used much less than “exclude”, because there are other ways to express those sentiments. Vetoing is specifically for objecting to a particular proposal being included *at all,* no matter how highly ranked other organizations might find it. It’s for expressing objection to a particular web technology, perhaps to something an organization has already expressed an objection to in the web standards process or otherwise has an objection to implementing. From fbc12e179a7dca0e7aefaa19ddd15257d7048358 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Wed, 11 Oct 2023 15:54:39 -0400 Subject: [PATCH 13/22] Update 2024/planning-process.md Co-authored-by: Chris Harrelson <3453258+chrishtr@users.noreply.github.com> --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index 3af6ba6..1e48581 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -82,7 +82,7 @@ In this round, each organization ranks the proposals in order of importance from Any proposal that gets a veto is immediately eliminated. -Each proposal gets assigned points from each organization, according to their rank, and an ordered list of all remaining proposals is created. Where `X` is number of proposals that are ranked, and `Y` is the ranking a particular proposal gets from a certain organization, then that proposal gets `X-Y` points from that organization. Once all 6 rankings are in for that proposal, the total points from all six organizations is known. If that proposal is not eliminated by a veto, then it has a certain number of points, and can be listed in order among the other proposals. +Each proposal gets assigned points from each organization, according to their rank, and an ordered list of all remaining proposals is created. Where `X` is number of proposals that are ranked, and `Y` is the ranking a particular proposal gets from a certain organization, then that proposal gets `X-Y+1` points from that organization. Once all 6 rankings are in for that proposal, the total points from all six organizations is known. If that proposal is not eliminated by a veto, then it has a certain number of points, and can be listed in order among the other proposals. For example, if there are 20 remaining proposals, each organization will choose a number 1, number 2, down to number 20. Let’s say a proposal for flying cars gets the following ranks: 1, 5, 7, 10, 14, 20. That will give it the following points: 20, 15, 13, 10, 6, 1 = 65 points total. From 2bbe6f9b0b7ffdd362abee45d34315d6eb359e97 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Wed, 11 Oct 2023 20:22:31 -0400 Subject: [PATCH 14/22] Update 2024/planning-process.md --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index 1e48581..24c6ca3 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -80,7 +80,7 @@ The official list of Focus Area proposals is updated to reflect the elimination In this round, each organization ranks the proposals in order of importance from most to least. And submits any vetos they may wish to exercise. The entire contents of this round is confidential to the team. -Any proposal that gets a veto is immediately eliminated. +Any proposal that has an objection is immediately eliminated. Each proposal gets assigned points from each organization, according to their rank, and an ordered list of all remaining proposals is created. Where `X` is number of proposals that are ranked, and `Y` is the ranking a particular proposal gets from a certain organization, then that proposal gets `X-Y+1` points from that organization. Once all 6 rankings are in for that proposal, the total points from all six organizations is known. If that proposal is not eliminated by a veto, then it has a certain number of points, and can be listed in order among the other proposals. From a5a4f1f22ec9f5f4ad25be92ddca2cc88cfae683 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:47:55 -0400 Subject: [PATCH 15/22] Update 2024/planning-process.md --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index 24c6ca3..ea30090 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -5,7 +5,7 @@ ### Timeline 1. Call for Proposal Period — three weeks (Sep 14 – Oct 5) -2. Continued discussion and refinement of proposals — one week (Oct 5 – Oct 12) +2. Continued discussion and refinement of proposals — one week (Oct 8 – Oct 14) 3. Elimination Round 1: Disqualification — one week (Oct 12 – Oct 19) 4. Elimination Round 2: Reduction — two weeks (Oct 19 – Nov 2) 5. Elimination Round 3: Prioritization — three weeks (Nov 2 – Nov 30, exc. North America Thanksgiving) From d95c7fe61d1d1eff51351571be0b3ca727382f77 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:48:08 -0400 Subject: [PATCH 16/22] Apply suggestions from code review --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index ea30090..c2762df 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -4,7 +4,7 @@ ### Timeline -1. Call for Proposal Period — three weeks (Sep 14 – Oct 5) +1. Call for Proposal Period — three weeks (Sep 14 – Oct 7) 2. Continued discussion and refinement of proposals — one week (Oct 8 – Oct 14) 3. Elimination Round 1: Disqualification — one week (Oct 12 – Oct 19) 4. Elimination Round 2: Reduction — two weeks (Oct 19 – Nov 2) From e1bb8e4e0d0f102751038753fdf48acab4411006 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:48:51 -0400 Subject: [PATCH 17/22] Update 2024/planning-process.md --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index c2762df..6969139 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -6,7 +6,7 @@ 1. Call for Proposal Period — three weeks (Sep 14 – Oct 7) 2. Continued discussion and refinement of proposals — one week (Oct 8 – Oct 14) -3. Elimination Round 1: Disqualification — one week (Oct 12 – Oct 19) +3. Elimination Round 1: Disqualification — one week (Oct 16 – Oct 20) 4. Elimination Round 2: Reduction — two weeks (Oct 19 – Nov 2) 5. Elimination Round 3: Prioritization — three weeks (Nov 2 – Nov 30, exc. North America Thanksgiving) From 9087fdeccee09b9358614f2e7813a04ed2fb504a Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:50:33 -0400 Subject: [PATCH 18/22] Update 2024/planning-process.md --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index 6969139..ee866df 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -7,7 +7,7 @@ 1. Call for Proposal Period — three weeks (Sep 14 – Oct 7) 2. Continued discussion and refinement of proposals — one week (Oct 8 – Oct 14) 3. Elimination Round 1: Disqualification — one week (Oct 16 – Oct 20) -4. Elimination Round 2: Reduction — two weeks (Oct 19 – Nov 2) +4. Elimination Round 2: Reduction — two weeks (Oct 23 – Nov 3) 5. Elimination Round 3: Prioritization — three weeks (Nov 2 – Nov 30, exc. North America Thanksgiving) Interop 2024 and the set of accepted proposals will be announced in January 2024. From 9ae26c39bcc0e69e38936550856b2a951f49a161 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:50:52 -0400 Subject: [PATCH 19/22] Update 2024/planning-process.md --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index ee866df..ba2f7a4 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -8,7 +8,7 @@ 2. Continued discussion and refinement of proposals — one week (Oct 8 – Oct 14) 3. Elimination Round 1: Disqualification — one week (Oct 16 – Oct 20) 4. Elimination Round 2: Reduction — two weeks (Oct 23 – Nov 3) -5. Elimination Round 3: Prioritization — three weeks (Nov 2 – Nov 30, exc. North America Thanksgiving) +5. Elimination Round 3: Prioritization — three weeks (Nov 6 – Nov 22) Interop 2024 and the set of accepted proposals will be announced in January 2024. From 52d94621dcd213c73c166f07cc8f7e2b0d336afb Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:51:12 -0400 Subject: [PATCH 20/22] Update 2024/planning-process.md --- 2024/planning-process.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/2024/planning-process.md b/2024/planning-process.md index ba2f7a4..c098f10 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -12,6 +12,8 @@ Interop 2024 and the set of accepted proposals will be announced in January 2024. +(This timeline can be adjusted as needed, at any time, by consensus of the Interop Team.) + ### Call for Proposal Period An open call for proposals will begin on a specific date, and run for three weeks. From 3d078cc32cd62ac8120e2ff9edb91148a75d5b54 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:51:39 -0400 Subject: [PATCH 21/22] Update 2024/planning-process.md --- 2024/planning-process.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/2024/planning-process.md b/2024/planning-process.md index c098f10..ffebb9f 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -14,6 +14,8 @@ Interop 2024 and the set of accepted proposals will be announced in January 2024 (This timeline can be adjusted as needed, at any time, by consensus of the Interop Team.) +(This timeline can be adjusted as needed, at any time, by consensus of the Interop Team.) + ### Call for Proposal Period An open call for proposals will begin on a specific date, and run for three weeks. From 45744aa03d901d73e543d4bae34a8fa0718c76a0 Mon Sep 17 00:00:00 2001 From: Jen Simmons <108474+jensimmons@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:55:23 -0400 Subject: [PATCH 22/22] Update 2024/planning-process.md --- 2024/planning-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2024/planning-process.md b/2024/planning-process.md index ffebb9f..781796f 100644 --- a/2024/planning-process.md +++ b/2024/planning-process.md @@ -24,7 +24,7 @@ An open call for proposals will begin on a specific date, and run for three week 2. Proposals should be for either: 1. Focus Area, or 2. Investigation Project -3. Related proposals should be filed separately. For example, all the ideas for “Web Compat” should be filed as a set of separate issues, not one issue. This makes it easier to discuss and prioritize each separately. Bundles of proposals risk being rejected or deprioritized. +3. Related proposals should be filed separately. For example, a group of specific features to address under “Typography” should be filed as separate issues, not one issue. This makes it easier to discuss and prioritize each separately. Bundles of proposals risk being rejected or deprioritized. 4. Focus Area proposals must: 1. Name a specific web technology. 1. This can be a web technology that has not yet been implemented in any or all browser engines.