Update our MEP infrastructure and documentation to reflect the MEP process

[choldgraf]

In our organization team compass we have the description of the MEP process that we currently follow. This is different than the process that is documented here. As we are getting close to having some MEPs merged I think we should update our docs here to reflect the new process.

Suggestions to act on:

• start this repo's commit history from scratch since the current MEP structure is much shorter
• copy paste the text at https://compass.executablebooks.org/en/latest/meps.html into a landing page with a brief overview of what MEPs are
• Set up the proper folder infrastructure to make our current MEPs mergeable
• delete the MEP page in the compass and cross link here
• we can then iterate from there but this at least gives us the base to start from

I'd also recommend using a theme that is part of this project, I think it's important that we use our own infrastructure and have a consistent brand across our docs sites, especially for core projects like MEPs. I'm fine w either the book theme / jupyter book, or mystjs though.

@rowanc1 any chance you'd have a moment to get to this while you're finishing up the cross references MEP?

[welcome]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.

If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).

Welcome to the EBP community! 🎉

[rowanc1]

Yep, I can take that on -- I will try to get to this today or tomorrow!

[chrisjsewell]

No I will do it thanks

[choldgraf]

@rowanc1 and I had a quick chat about this earlier, so I'd like him to take the lead here. If he wishes to delegate to you on this one that is fine w me though 👍

[chrisjsewell]

start this repo's commit history from scratch since the current MEP structure is much shorter

Ermm, I feel this is quite in bad faith, and don't know it fits in with the code of conduct of this organization?

You are talking about wiping out peoples contribution to the project, namely all my initial commits to set up this repository

I would note also, that in this commit: executablebooks/meta@0377df6#diff-e9a11e85a944c43b1bf3dd2cdc09352dc016824279958134e45e91dbc16488c1L22

you have already previously, removed the acknowledgement of the work I did here:

I hope you can see why I'm a little unpleased with this approach?

[chrisjsewell]

I'm absolutely happy to work with @rowanc1 and you on this, but yeh well "rewriting history" is a little much for me to take 😬

[choldgraf]

Agreed that we should provide credit to those that participated in the MEP process bootstrap. Your initial commit in this repository was appreciated and a helpful contribution to our initial process.

Something that @rowanc1 and I discussed was cross linking that PR as part of the documentation here, and listing the team members that took part in that discussion or gave feedback on the draft. That way we credit the team of people that were involved. I think it's important to recognize the many voices that went into the MEP process discussion.

[chrisjsewell]

The sentiment is appreciated, but still I don't see a good argument for wiping a commit history of a repository, I have never seen that happen in any project.

[choldgraf]

Re: rewriting history, the current state of main was not broadly discussed or approved, and was not an official process, so I think we should treat it as draft language. History in this repository should "begin" with the first official MEP process that was created by the team and approved by the steering council.

I'd suggest we rename main to a new branch and cross-link that branch from the PR that we link in the MEP documentation. Then we create a new main and iterate from there. That way history isn't lost, your draft language is available for provenance, and we begin this repository history with the official MEP process that was defined.

[chrisjsewell]

I somewhat bemused by this sudden insistence that history in a repository should have an "official" start, but in any case...

copy paste the text at compass.executablebooks.org/en/latest/meps.html into a landing page with a brief overview of what MEPs are
That way we credit the team of people that were involved.

I suggest that this this text should be MEP0001, as is the case now (and for https://peps.python.org/pep-0001/, https://jupyter.org/enhancement-proposals/29-jep-process/jep-process.html, etc), and those involved credited as the authors

[choldgraf]

Yes I was thinking the same - we should treat the PR in meta as the "proposal + discussion" for MEP001

[chrisjsewell]

Yes I was thinking the same - we should treat the PR in meta as the "proposal + discussion" for MEP001

ok cool, if I'm on the author list for that, then we have a deal lol, and I'm not so much bothered about the commit history 😅

and again, if you do need any help than let me know, I am happy to assist

[choldgraf]

For the "bootstrap MEP", my thinking was that everybody that participated with edits, comments on draft language, or discussion in the PR will be on the author list in alphabetical order, or that we just have a single author of "The ExecutableBooks Team".

Not sure which is better given that there are no formal authorship guidelines. For subsequent MEPs we have a formal author field so this will be more explicit.

[rowanc1]

@choldgraf do you think you can take this on please. At least the initial MEP0001 commit -- I can help out once that is over the line.

For what it is worth, I am a fan of The ExecutableBooks Team, especially for the first MEP, as there were other people who also reviewed drafts and did not participate directly in the github discussions.

[chrisjsewell]

The team could change over time, I think it should be everyone in alphabetical order

[chrisjsewell]

I would note, and this is the last thing, I'm probably reacting to this so strongly, because I have already been biting my tongue on another incident where attribution of effort seems to have slipped: jupyter-book/mystmd#184

Here, essentially the entirety of the code I mainly wrote in markdown-it-docutils was copy-pasted into mystjs, without any mention

Its not the end of the world, but just a quick "hey @chrisjsewell just to let you know we are doing this because we want to integrate the code better, ..." would have been nice, rather than me having to find out second hand, as I imagine would have been the case here if I was not subscribed to the issues 🤷

I didn't mention, because I didn't want there to be negativity, but when it starts to become a trend

[mmcky]

It was my understand that this repo was in draft as we knocked around ideas on how get the mep process up and running.

@chrisjsewell we all know you contributed extensively to this process. I fully expect it would be documented as part of this process as @choldgraf has pointed out.

FWIW - While I agree it is not common to reset history in a software repo -- I am in favour of reseting the history in this non-code repo case. I don't think it is always helpful to record everything in micro detail until the process is up and running and actively used -- in fact the opposite -- it is better not to have the history to bring focus to an agreed process.

[chrisjsewell]

it is better not to have the history to bring focus to an agreed process.

I honestly doubt anyone will be looking at the commit history to understand the process,
but anyhow, that's not really the primary thing here.
I hope you can see why it might be demotivating to happen across an issue that says your commits are going to be wiped, without any mention to you.

I fully expect it would be documented as part of this process

and yes, also with #14 (comment),
where the acknowledgement was removed only a month after it was added,
I hope you can also understand that it might be difficult for me to trust that any such documentation would persist 😞
(as opposed to as an author of an MEP, which is a little more binding)

[choldgraf]

Sorry I didn't get to this yesterday, I ended up being pretty sick the last two days, I'll try to get to this today. Two quick responses to the above:

I hope you can see why it might be demotivating to happen across an issue that says your commits are going to be wiped, without any mention to you.

I can understand this. In the future the way to avoid it is to propose new processes via getting agreement from team members before taking action. Create a draft proposal (e.g., an issue, a hackmd, a google doc, etc), get a coalition of team members to support and advocate for it, make a decision w/ the team or steering council if necessary, and then implement something. (basically our new policy making guidelines). By aligning the team before implementation, we can avoid the case that a lot of work by one person is undone after a decision is made post-hoc.

and yes, also with #14 (comment),
where the acknowledgement was removed only a month after it was added,

I removed that admonition because it's purpose was to provide informal context/acknowledgement in the discussion process when we were merging the PR, but was not meant as a long-term attribution since we do not use our meta/ documentation to provide attribution for any other specific content. For major updates to MEP process, the appropriate place for attribution is via the authorship list in the MEPs rather than in the implementation that exists in the docs. This is why you will be part of the authors list of MEP001, and why I removed that section from the meta/ docs. I did not provide attribution to myself in the team compass docs for the same reason.

[chrisjsewell]

Thanks for the response @choldgraf, that all sounds good. Hope you are feeling better!

[choldgraf]

Draft is up

Hey all - I've got a draft of the docs up at this branch:

https://github.com/executablebooks/myst-enhancement-proposals/tree/copy-from-meta

There are still a few things I'd like to add, like that nifty "markdown table insert" that @chrisjsewell implemented. I want to do that as a separate commit w/ him as the author though, since I'm largely just copypasting, but don't have time to figure it out right now so will get to that tomorrow.¹

Would love any feedback on the docs there, I think it's fine to iterate on all of this stuff in small chunks as long as we don't make any major changes to the process.

Footnotes

1. by the way, this made me think it might be useful to have that little "show the page metadata" transform as a page-level flag that comes with the myst parsers (e.g. :myst.show_metadata or something? or a directive where you could also provide a list of keys to include or exclude?). I think many projects might use this for structured sub-sections of documentation (like enhancement proposals) ↩

[choldgraf]

Made the switch on main

I've updated main with the new branch, and the docs are now here:

https://myst-eps.readthedocs.io/en/latest/

question: Do we want to rename the website to myst-enhancement-proposals as well? Or leave it as-is?

The previous main is now this branch: https://github.com/executablebooks/myst-enhancement-proposals/tree/chrisjsewell-mep-process-proposal . I've also cross-linked the HackMD that was originally used for it in the MEP0001 and in the top comment of the discussion PR. I can update that to something else if that's not the right link to use.

cc @rowanc1 who has a few PRs you'll need to update.

I'll let you all decide how you'd like to proceed with #5, but my suggestion is to hold off for a while until we have a few more targeted MEPs under our belts and a tighter process for discussion etc.

@chrisjsewell can you add others to the readthedocs project? At a minimum I would say me and @rowanc1 since we're all the ones participating in this bootstrap implementation right now.

[chrisjsewell]

yep looks good cheers 🎉

@chrisjsewell can you add others to the readthedocs project?

Done 👍

There are still a few things I'd like to add, like that nifty "markdown table insert" that @chrisjsewell implemented.

I'm a little confused, so that is already there now?

[chrisjsewell]

by the way, this made me think it might be useful to have that little "show the page metadata" transform as a page-level flag that comes with the myst parsers (e.g. :myst.show_metadata or something?

yep something like that could be handy indeed

[choldgraf]

Yep it's added now w you as the commit author, i did that this morning (i hope that's ok, it was basically just copy paste and tweak to match new key names)

[chrisjsewell]

hope that's ok, it was basically just copy paste and tweak to match new key names

yep all good

[rowanc1]

I have updated the two PRs, and rebased them on main.

Do we want to rename the website to myst-enhancement-proposals as well? Or leave it as-is?

I think that we should bring this under mep.myst-tools.org.

@chrisjsewell my readthedocs username is rowanc1 if you want to add me there please.

Exciting to see this process and website coming together. Thanks all! 🚀

[chrisjsewell]

my readthedocs username is rowanc1 if you want to add me there please.

Already done