Reviewed JEP process

[JohanMabille]

This is an update of the JEP process. Main changes are:

• the Shepherd is now optional
• the voting phase has been explicitly added
• An intermediate status has been added to help JEP triage and review
• the JEP process diagram has been updated according to these changes.

After this is reviewed and merged, we should make clear that JEP 29 is deprecated or outdated to avoid confusion (since there will be two JEPs describing the process). Another solution would be to open a PR to the JEP 29 that has already been merged instead of this new JEP, but I don't like this solution as it "erases" the history.

There is also an old description of the JEP process here that we should replace with the last version of the JEP process (i.e. the doument from this PR when it is approved).

Looking forward to reading your feedbacks!

---

Voting from @jupyter/software-steering-council

• @echarles
  • Yes
  • No
  • Abstain
• @fcollonval
  • Yes
  • No
  • Abstain
• @ibdafna
  • Yes
  • No
  • Abstain
• @isabela-pf
  • Yes
  • No
  • Abstain
• @ivanov
  • Yes
  • No
  • Abstain
• @JohanMabille
  • Yes
  • No
  • Abstain
• @minrk
  • Yes
  • No
  • Abstain
• @rpwagner
  • Yes
  • No
  • Abstain
• @SylvainCorlay
  • Yes
  • No
  • Abstain
• @Zsailer
  • Yes
  • No
  • Abstain

[manics]

Here's the diff from https://github.com/jupyter/enhancement-proposals/blob/master/29-jep-process/jep-process.md for comparison:

--- jep-process.md	2023-04-04 10:41:10.036810233 +0000
+++ jep-process-v2.md	2023-04-04 10:41:39.927751732 +0000
@@ -70,12 +70,12 @@
 1. A github issue on the Jupyter Enhancement Proposals repo is created that
 2. 1. Briefly outlines the proposal
 
-   2. Suggested review team (optional)
+   2. Suggests reviewiers (optional) in addition to the SSC members.
 
    3. Why it should be a JEP
 
    4. 1. See the “JEP / Not-a-JEP Rubric” below.
-3. A *Shepherd* is identified to see the process through. (Shepherds are assigned on a round-robin basis from a set of interested engaged volunteers).
+3. A *Shepherd* is identified to see the process through. Shepherds are assigned from a set of people actively involved in the project (Sotfware subprojects, SSC or EC, Standing committees and Working groups), based on the scope of the JEP. The Shepherd can be the author of the JEP.
 4. A number is assigned to the JEP to track it through the rest of the process.
 
 Outcome:
@@ -87,9 +87,11 @@
 
 ### Phase 2: RFC for the JEP
 
-Submission: The author submits an initial draft of the JEP as a PR to the JEP repository starting from the relevant template decided in the pre-proposal stage. The Shepherd assigns a number (the GitHub PR number? A sequential number? Perhaps the number isn't assigned until the JEP is merged?). The Shepherd assigns a Review Team.
+Submission: The author submits an initial draft of the JEP as a PR to the JEP repository starting from the relevant template decided in the pre-proposal stage. The number assigned to the PR by Github is the JEP number. The Shepherd optionally assigns reviewers in addition to the SSC. Reviewers and the SSC are refered as the Review Team in the following.
 
-Request For Comment (RFC) phase: The proposal is iterated on with feedback from the Review Team and the community at large. The Shepherd helps engage the Review Team. After the Review Team members have signed off on the JEP, with the criteria that there are no major objections, and at least some of the Review Team are in favor, the Shepherd initiates the Final Comment Period.
+Request For Comment (RFC) phase: The proposal is iterated on with feedback from the Review Team and the community at large. The Shepherd helps engage the Review Team. When the proposal matures, if there are no major objections, the Shepherd calls to a vote from the SSC members.
+
+Waiting Decision (WD) phase: The vote follows the rules described in the Decision-Making Guide of the Jupyter Governance. The SSC members have seven days to vote, although the council may consider longer voting periods. If the vote results in an acceptation of the JEP, the Shepherd initiates the Final Comment Period. Otherwise the JEP is considered as rejected.
 
 Final Comment Period (FCP): The community at large has 10 calendar days in which to provide any final comments on the JEP. A JEP may revert back to RFC if objections are supported by the Review Team. If not reverted to the RFC phase, the JEP is approved at the end of the FCP.
 
@@ -108,7 +110,19 @@
 
 ### Paths of the status of JEPs
 
-![img](https://docs.google.com/a/safia.rocks/drawings/d/s9fsfcbWM0mamBZBDJpK0sA/image?w=583&h=255&rev=154&ac=1&parent=16xa1DSH0lN6lY-EzyyGOzaZlcQc1ZT8HEeDc3uyrwcI)
+![img](jep_status.drawio.svg)
+
+- Pre-proposal: first stage of the JEP
+- RFC: Request For Comments, proposal is under active discussion and revision
+- Waiting for answer: authors have been pinged and we will wait 2 weeks before revising the status
+- Waiting for decision: final decision to approve or not to be sanctioned by SSC
+- FCP: Final Comment Period, 10 days period for any final comment before approval
+- In progress: JEP has been approve, implementation is on progress
+- Completed: Implementation has completed
+- Withdraw: at any stage the author cn decide to withdraw the JEP
+- Deferred: Inactive draft that may be taken up again at a later time
+- Rejected: The JEP has been rejected and will not be implemented
+
 
 ## What qualifies as a JEP?
 
@@ -148,27 +162,3 @@
 Note that the JEPs themselves contain the content, while the website is just a
 quick way to display them in a reading-friendly format.
 
-
-## Glossary
-
-- **Jupyter Enhancement Proposal (JEP)**: A written document that describes a proposed unit of significant work on Jupyter.
-
-- **Contributor**: The person (or persons) who is submitting the JEP. The implementation of a JEP may be done by others if so approved.
-
-- **Shepherd**: A senior Jupyter contributor who guides the **JEP Contributor** through the pre-proposal, submission, review, approval process of a JEP.
-
-- **Review Team (RT)**: A group of Jupyter contributors, with expertise in a particular area of the project, that reviews JEPs related to that area.
-
-- **Final Comment Period (FCP)**: A finite-time, pre-approval period for the community to make final comments.
-
-- JEP Statuses:
-
-  - Inactive
-  - Submitted
-  - Assigned
-  - Rejected
-  - Postponed
-  - Withdrawn
-  - Approved
-  - In progress
-  - Completed

[fcollonval]

Suggested change

	3. A *Shepherd* is identified to see the process through. Shepherds are assigned from a set of people actively involved in the project (Sotfware subprojects, SSC or EC, Standing committees and Working groups), based on the scope of the JEP. The Shepherd can be the author of the JEP.
	assigned from a set of people actively involved in the project (Sotfware subprojects, SSC or EC, Standing committees and Working groups)

As this restricts the people eligible as Shepherd should we have a fallback like the SSC will pick a shepherd or clarify how the shepherd is chosen. My concern is that if a JEP author is not well known member of the community, he/she may not understand what to do to find a shepherd.

[JohanMabille]

I agree, I'll add this fallback.

[ivanov]

I pushed up some minor fixes, and on-board with making these changes, but how do we feel about adding a "Superseded" or "Replaced" status for JEPs? It seems like this is exactly the sort of thing we would want to apply to JEP29 as part of this.

Another alternative would be to have an "Active" status, which is what Python does with PEPs that can change and be updated, like PEP 1.

Here's the process flow from PEP 1, for reference.

[ivanov]

maybe add a "Date updated" here for this revision of the changes.

[minrk]

how do we feel about adding a "Superseded" or "Replaced" status for JEPs?

I think both 'superseded' and 'active' states make sense, thanks for bringing those up!

If we have 'active' for process JEPs like 29, we may not need 'superseded' specifically as a state, and can do with adding a note that folks looking at an accepted JEP may want to look at a later one. A formal 'superseded' state may not really solve a problem.

[JohanMabille]

I finally took the time to get back to this; I've taken into account the last comments and reworked the JEP workflow. If everyone is happy with it, we could call a vote by the end of the week.

[rpwagner]

The example of creating a single repo within an existing GitHub org feels too specific for a JEP. However, the creation of a new GitHub org does need a more well-defined process to ensure that it has the appropriate ownership and configuration (e.g., 2FA). I suggest changing this line to simply Creating a new official GitHub organization.

Thinking further, it may be worth a JEP to clarify the creation and management of GitHub entities (orgs, repos, teams, etc.). I think the SSC is the place to handle that JEP, with additional review by the Governance Council.

[JohanMabille]

The example of creating a single repo within an existing GitHub org feels too specific for a JEP. However, the creation of a new GitHub org does need a more well-defined process to ensure that it has the appropriate ownership and configuration (e.g., 2FA). I suggest changing this line to simply Creating a new official GitHub organization.

I agree, and the way we have done in recent years tends to confirm that.

Thinking further, it may be worth a JEP to clarify the creation and management of GitHub entities (orgs, repos, teams, etc.). I think the SSC is the place to handle that JEP, with additional review by the Governance Council.

Totally agree on this! However, I think it should not block this JEP, we can amend it later when the JEP specifying the creation and management of GitHub entities is done.

[rpwagner]

Totally agree on this! However, I think it should not block this JEP, we can amend it later when the JEP specifying the creation and management of GitHub entities is done.

Absolutely should not block this one! I was just thinking out loud (or in writing).

[ivanov]

I see a bunch of checkmarks already, @JohanMabille are we moving onto the voting period? if so, let's ping the SSC.

[JohanMabille]

@ivanov indeed, I think all the concerns have been addressed. I thought editing the first comment for calling for a vote would ping everyone, it didn't work?

[ivanov]

I thought editing the first comment for calling for a vote would ping everyone, it didn't work?

indeed it did not (I was just notified of changes here from prior participation)

@jupyter/software-steering-council - this proposal has moved onto the voting phase.