Initial Documentation working group charter

[RRosio]

This is the Documentation working group charter, we are ready for feedback!

---

Documentation Working Group Charter
Initiating a vote on a new Working Group Charter

This is the charter for Documentation Working Group.

Please vote on this charter by Friday, January 12th, 2024 EOD:

Executive Council:

• @afshin
  • Yes
  • No
  • Abstain
• @ellisonbg
  • Yes
  • No
  • Abstain
• @jasongrout
  • Yes
  • No
  • Abstain
• @fperez
  • Yes
  • No
  • Abstain
• @Ruv7
  • Yes
  • No
  • Abstain
• @blink1073
  • Yes
  • No
  • Abstain

[blink1073]

Some of these seem a bit too technical and specific to be included in the charter, I'd suggest being more high level and tool agnostic.

[chbrandt]

How about this @blink1073 ?

• This means, for members of the group, also taking on any related (non-document) tasks that will be beneficial to docs and support the Docs Working Group's goals, such as:
  • Improve the user experience in Jupyter applications through offline or interactive documentation extensions;
  • Implement analytics for (ReadTheDocs) documentation webpages for better understanding of docs use.

[ericsnekbytes]

I think the important part here is "This means...also taking on any related (non-document) tasks", to explicitly call out that our scope includes development work and other non-writing work that supports the mission. Most of these items are intended to be clarifying examples that illustrate the intended meaning. I do take your point about being tool-agnostic though... @chbrandt I think that looks good.

[jasongrout]

Just to call it out explicitly, there is more context to this PR at the EC team-compass issue: jupyter-governance/ec-team-compass#19

[afshin]

I don't have an objection to this at all, but I'd remove it from the charter and instead create an issue on the docs working group repo and handle it as a task instead of a chartered mandate.

[ellisonbg]

I love these pillars :-) We have needed this in the community for a long time, really appreciate everyone who is stepping up to help.

[ellisonbg]

In the Jupyter ecosystem, the term "doc" has a number of different meanings (notebook docs, documentation, etc.). It may be helpful to spell out "Documentation" in the charter to clarify this to people who may not have full context.

[ellisonbg]

One question that I have is that Jupyter has some documentation that isn't (currently) part of subprojects. Will this WG own that documentation? Right now this type of documentation is in the grey area of the project where it isn't clear who can even review/merged PRs. I think it would be helpful if the charter gave the WG ownership of those docs.

[ellisonbg]

There maybe room for two sections here (responsibilities and activities). For some things I think the WG should own and have a responsibility for something. For other things, their role is more of a helper for subprojects where activity is maybe a better term.

[ericsnekbytes]

Pinging founding members, this latest commit/version is closest to what we're going to submit @chbrandt @willingc @fcollonval @krassowski @bollwyvl @ivanov @RRosio.

[ericsnekbytes]

The changes have been added, and we're ready for review!

[RRosio]

Call for vote has been moved to top-level comment.

[choldgraf]

Hey all - I just want to voice my support for this working group! I didn't see earlier notifications around its founding and I appreciate the effort folks made to ping me :-). Mostly I just have very little time these days because I have 2 really young kids at home. But documentation is near-and-dear to my heart so let me know what I can do to support you all.

For what it's worth, I think that this is an important gap in Jupyter's capacity and sharing resources and knowledge around documentation is a great idea. I added a few comments throughout - mostly around clarifying what "own" means in practice. For shared documentation that touches all of the sub-projects (like jupyter.org) I've often found tension comes up when there is not clarity about who needs to be consulted vs. informed when describing the project.

If I can be useful on the infrastructure side of things (e.g. the pydata theme, myst markdown in Sphinx, the new MyST build engine at mystmd.org, etc) then I'm happy to help. I have limited time for sustained engagement over time (you can thank my 2 kids) but I will do what I can!

[choldgraf]

I'd suggest clarifying what "takes ownership" means. Does this mean operational ownership, or content-level ownership? If content-level ownership are there certain kinds of content that require consultation or approval from other parts of Jupyter before they're changed?

[ericsnekbytes]

Thank you!

Based on feedback from the EC during the weekly meeting, we added these two to make sure our powers are broad enough to do what makes sense without needing to return to the EC to ask for additional powers.

The Docs Working Group is cross-functional and collaborative down to its bones, so we expect to tread VERY lightly with any binding standards (and frankly I don't think standards will be needed in the foreseeable future).

For taking ownership of jupyter.org and other orphaned docs/resources (as suggested in our meeting with the EC), we would take on both operational and content-level ownership where there's not already someone to tend to them. There seemed to be pretty broad agreement during the meeting that jupyter.org docs would be a good place for "cross-project" or "project wide" information about Jupyter, ALSO including contributor and maintenance info for the project for those working at the project-wide level.

If we need to modify these, we can add some extra detail to clarify.

[ericsnekbytes]

And for the jupyter.org docs, the Docs Working Group would be building upon what's already there (and the people/procedures already in place) and expanding where we need to, to follow this vision of the site/docs as a common place for project-wide information and contributor guides/materials that the whole project can (re)use and reference where needed.

[choldgraf]

Makes sense - I am all for the vision, just trying to provide some experiences over the years watching the jupyter.org docs be stewarded by various folks. I think the most important thing to remember is that jupyter.org has a large reach, and so it is effectively a way to massively signal boost certain parts of the project.

With that in mind, a lot of the tension has come from questions like "which sub-projects get to have a top-level navigation item?" So just keep that kind of dynamic in mind and plan-ahead for thoughtful decision-making frameworks to navigate those topics! I don't think it's anything insurmountable, but just good to front-load some decision-making and process infrastructure so the docs working group doesn't get caught in a bind unnecessarily.

[Zsailer]

echoing @choldgraf's concern here.

jupyter.org has been a touchy subject in the past. Because its the most broad-reaching page in Jupyter, it's precious real estate.

Today, it's not exactly clear which group gets final say on what is added/removed from this page. The EC manages the jupyter.org service (based on the governance docs), but I don't know if that means the EC is the final decision-maker on its content.

Making the Doc's WG the group that "owns" jupyter.org, as it is worded here, seems like a big move that requires further discussion. Is this really the intention? Or are we just moving the "management", i.e. ensuring it's running and maintaining its current status, the intention?

and expanding where we need to, to follow this vision of the site/docs as a common place for project-wide information and contributor guides/materials that the whole project can (re)use and reference where needed.

This is precisely the "touchy" point. Expanding the site is often met with debate/friction because Jupyter.org is precious real-estate.

As an example, see the discussion on this PR previously around the top Nav Bar: jupyter/jupyter.github.io#321 (comment)

Deciding what should/should not be listed on this site might need more oversight than this working group...

[Zsailer]

Also, its worth mentioning, there has been a JEP open for a many years that proposes to re-design jupyter.org: jupyter/enhancement-proposals#49

The reason for this proposal was that jupyter.org was—after doing a bunch of research, user interviews/testing, and experimentation—a UX team (we temporarily used a Master's program team) found that jupyter.org was becoming too busy that it was losing its utility. Building this website so that it includes all the necessary docs/information for the project while remaining intuitive is actually quite hard to achieve. The re-design was a solid effort at improving this.

This would be a good thing to revisit in the working group, if the Docs WG takes over jupyter.org.

[choldgraf]

Are standards "binding" for sub-projects or are they more like shared resources that sub-projects can use if they wish?

An example - in the JupyterHub project we have a shared modification of the PyData Sphinx Theme defined here:

https://github.com/jupyterhub/jupyterhub-sphinx-theme

This allows JupyterHub projects to use the theme in order to standardize the look and feel across sub-projects. But it's not a "requirement" - it's just a shared resource to be used as is useful.

[ericsnekbytes]

The vision I have for this: The PyData Sphinx theme could be further developed by the Docs Working Group, in consultation/collaboration with the JupyterHub team and existing devs, then we could take point on advocating for it, both as a resource to teams that need it, and as a standard to make things consistent across the ecosystem.

This collaborative pattern is at the heart of how the Docs Working Group will operate (we help, we do work, we collaborate to understand needs and get feedback, we build resources that other teams can use, and we advocate for their usage to subprojects in accordance with our mission to make the Jupyter ecosystem better and more consistent).

If we do our work right, we will have gathered needs from the subprojects that inform what the theme is capable of, so that they have what they need for that resource to make sense for their project.

[choldgraf]

I'd love to see this working group be a source of more focused development (or even just specific requests) in the PyData theme (or the sphinx book theme). Those are both chronically under-supported and maintainer-led projects!

If we do our work right, we will have gathered needs from the subprojects that inform what the theme is capable of, so that they have what they need for that resource to make sense for their project.

I like the idea of framing this group as doing a bit of "user research and product development" where the users are Jupyter maintainers that want help with their docs, and the product is infrastructure that makes it happen.

[fperez]

Thanks everyone for this!! Reviewing today, will provide feedback and/or vote later, just pinging here so everyone knows we're not ignoring it :)

[ericsnekbytes]

Does anyone object to moving the deadline to vote up two weeks (to Friday Jan 12th) because of the holidays?

[Ruv7]

Moving the deadline is best, thanks @ericsnekbytes

[ellisonbg]

When the people involved in this WG proposal joined the EC office hours, I believe we discussed that the WG would own all documentation not part of subprojects, including the documentation related parts of jupyter.org, but I don't believe we talked about the WG owning all of jupyter.org. I don't think that makes sense as the website has a ton of content that is unrelated to documentation (sponsorship, "marketing", messaging, fundraising, governance, etc.).

[ericsnekbytes]

It's clear to me that we need to make some further edits to the charter, and to clarify some points of confusion (my own and others) around some related issues.

Right now, the charter says (under "Areas of Responsibility"):

"The Docs working group will take ownership of documentation and related resources not owned by a Subproject (including the Jupyter.org website and documentation)."

This was based directly on @ellisonbg's feedback included here, below:

"One question that I have is that Jupyter has some documentation that isn't (currently) part of subprojects. Will this WG own that documentation? Right now this type of documentation is in the grey area of the project where it isn't clear who can even review/merged PRs. I think it would be helpful if the charter gave the WG ownership of those docs."

We can revise the charter to say The Jupyter.org documentation sites (under the current 'Documentation' header item) or similar, which should help narrow the scope and meaning of that section. Do we have consensus on this narrowed clause? I think it makes sense for the Docs Working Group to own that, and it should be close to the meaning of @ellisonbg's original suggestion.

Beyond that, it was originally my (seemingly wrong) impression that the larger Jupyter.org site was another mostly orphaned Jupyter asset (I'd heard in the weekly meetings that the site needed refactoring but that nobody was available to do it). So it seems my impression was wrong, and that the EC owns the Jupyter.org site and has existing procedures/resources around content and maintenance...is that accurate?

I'm comfortable removing the larger "Jupyter.org website" from our charter and sticking with the narrower Jupyter.org documentation sites (under the current 'Documentation' header item).

[ericsnekbytes]

Some additional thoughts about this thread and @Zsailer’s comments above:

Jupyter (the project) is home to many key pieces that are critical to its success.

Is Server more or less important than Notebook? Or Lab, or Hub? That's a ridiculous question, akin to picking the heart over the brain, or kidneys over the lungs. They're all important, and add different things that are important to different stakeholders (contributing to the overall success and health of the project).

At the same time, none of these pieces is MORE important than the project as a whole. We give each Subproject a lot of independence and autonomy, which is part of what makes our collaborative open source work culture so successful (and each subproject so robust/vibrant). But when stakeholders from different Subprojects feel like they’re competing (for space on the home page for instance), it’s detrimental to the project as a whole.

If everyone is shouting at the top of their lungs, nobody is heard. The listener (or reader), instead of taking away a few limited but important pieces of information about Jupyter, is left with a jumbled incoherent mess of competing minutiae.

In my mind, this (fragmentation) is why we keep hearing the same confusion/feedback from users, described well by @rpwagner via security stakeholders in the original Docs Working Group proposal thread:

…one of the repeated pieces of feedback we've heard from groups deploying Jupyter components (JupyterLab, JupyterHub, etc.) who aren't active in the community is that it's challenging to get a unified view of "what is Jupyter" and how the components work together.

This is why we need Jupyter-wide groups like the Docs Working Group, to unify and harmonize what has become a very fragmented, confusing experience for our users.

To that end, in my mind, people who come to Jupyter.org should understand that:

• The Jupyter project has many key pieces, all about interactive computing with computational notebooks
• You can customize, improve and extend Jupyter software to do just about anything you need

If we can’t agree on these or a similar simple, clear communication hierarchy of ideas about Jupyter to communicate (rather than having subprojects all competing to cram the universe into the first sentence), our users will remain lost and confused.

[Zsailer]

We can revise the charter to say The Jupyter.org documentation sites (under the current 'Documentation' header item) or similar, which should help narrow the scope and meaning of that section. Do we have consensus on this narrowed clause?

Yep, this phrasing looks good to me.

BTW, don't get me wrong. I'm not against the WG having more decision-making power on jupyter.org; I just think that's a bigger change than originally intended—so I wanted to flag it.

the EC owns the Jupyter.org site and has existing procedures/resources around content and maintenance...is that accurate?

That's my point above; I don't know if this is true. Here's what the governance docs say about the EC:

The EC "manages" the jupyter.org service; but I interpret that to mean they ensure that the domain is always available and hosted appropriately. They don't have sole ownership over what content gets posted there.

Up until this point, content was always been decided by "consensus", but as the site has grown, consensus has been more difficult to obtain.

I'd heard in the weekly meetings that the site needed refactoring but that nobody was available to do it

I don't think it's an issue of availability; rather, it's the consensus problem.

If we can’t agree on these or a similar simple, clear communication hierarchy of ideas about Jupyter to communicate (rather than having subprojects all competing to cram the universe into the first sentence), our users will remain lost and confused.

I think most (if not everyone) agrees with this sentiment, but it's difficult in practice. There are many different answers to the question "what is Jupyter?". In fact, there is a whole class of "Jupyter users" who have never touched a computational Notebook. They use kernels and nothing else—in fact, Jupyter started as an interactive shell/console 😉

Interactive computing is definitely a core theme; but that's likely not concrete enough.

That said, definitely take a look at the JEP for a re-design; I think it attempts to capture the docs experience you're describing and you'll appreciate. I'd love to see someone pick that effort up again and push it through. Even if we didn't want to use the design assets proposed there, we could learn a lot from the logical hierarchy of the site they proposed.

But when stakeholders from different Subprojects feel like they’re competing (for space on the home page for instance), it’s detrimental to the project as a whole.

I think there's subtle difference from the picture you describe here and what I've seen happen.

It's often not the developers of the subprojects demanding a higher position on the "Jupyter org" chart; the confusion stems from our attempt to document (and failure to agree on) what users look to Jupyter to provide.

Our user community is diverse when answering the question "What is Jupyter?" or "what does Jupyter mean to you?". e.g. some think Lab, some like Notebook. Some think IPython shell, some think interactive webpages (Voila/Lite/etc). Some think Cloud, some think localhost. Some think web app, others think desktop. Some think protocols/standards (in fact, that's what won Jupyter the ACM award...)

[ericsnekbytes]

@Zsailer The redesign JEP seems to pretty successfully communicate the diverse offerings and use cases of Jupyter products :) And the Docs WG can definitely take the lead on the Jupyter.org redesign/JEP. It's a natural fit because it's an important resource that helps the outside world understand the Jupyter ecosystem (which is tied to the Docs WG's mission).

I've observed the difficulties obtaining consensus as I've onboarded to Jupyter work these past ~2 years: We're often left stalled on important work, or as is often the case with docs- or communication- related efforts, we end up with outputs that don't communicate effectively because too much has been included, and what's there has not been structured according to a coherent underlying vision of what concepts are most important (and I hope to develop some of these in collaboration with all the subprojects).

Jupyter-wide challenges like the consensus problem are likely to grow over time in step with the project's growth. I'm hoping the Docs WG can help innovate solutions for working through these type of organization-wide problems (maybe there are some reusable work processes to discover and share here).

These issues, in my mind, make a clearer case for the creation of new Jupyter-wide bodies (like the DWG) that act to add consistency, unity and cohesion to Jupyter offerings (and to counteract any fragmentation that arises between subprojects).

By ensuring these new groups are highly collaborative and deeply connected to the subprojects, and by acting from a project-wide perspective that takes the health of the overall project as a priority, my hope is that we can more effectively solve these sorts of systemic problems.

[Zsailer]

These issues, in my mind, make a clearer case for the creation of new Jupyter-wide bodies (like the DWG) that act to add consistency, unity and cohesion to Jupyter offerings (and to counteract any fragmentation that arises between subprojects).

💯 🚀 I love it, @ericsnekbytes! Thank you for taking this on—you've got my support!

[RRosio]

Hi everyone, we have made a small edit to the charter's Jupyter.org section as was discussed above. Please remember to vote by Jan. 12th 2024, thank you!

[ericsnekbytes]

Hello everyone, does anybody have additional concerns that we can address? I want to make sure we've hit any remaining points, and am trying to take the temperature of the group to see if we're on track for passage or not. Thanks for all of the support and discussion so far!

[ericsnekbytes]

Quick reminder: last day to vote!

[ellisonbg]

Thanks for the reminder, I have pinged Fernando and Ana.

…

On Fri, Jan 12, 2024 at 8:26 AM Eric Gentry ***@***.***> wrote: Quick reminder: last day to vote! — Reply to this email directly, view it on GitHub <#198 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAGXUFDFPD25WSW5V5ML7LYOFIZJAVCNFSM6AAAAABATT6CT2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBZGUYDMNRUGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Brian E. Granger Senior Principal Technologist, AWS AI/ML ***@***.***) On Leave - Professor of Physics and Data Science, Cal Poly @ellisonbg on GitHub

[fperez]

Done folks, sorry for the delay! A combo of unexpected travel/medical stuff made me drop a lot of balls, appreciate the ping by @ellisonbg.

I want to thank everyone for the work - I'm very excited by this development which I think we've needed for a long time in our community, and will provide a ton of benefit. Much appreciated, happy weekend everyone!

[ericsnekbytes]

With a quorum and 5/6 Yes votes, it looks like the Docs Working Group charter is approved! Assuming I haven't overlooked anything, can someone merge this in?

Thank you! To everyone involved in bootstrapping the Docs Working Group, in particular @chbrandt, @RRosio, and @willingc, as well as @afshin for providing crucial guidance, and @bollwyvl, @fcollonval and @krassowski for providing important early feedback.

We have much to do! I'm looking forward to making tangible improvements to docs across the Jupyter ecosystem, with the help of my fellow council mates, the subprojects and the community :D

I will be working up some community announcements in the meantime. Cheers!

[fperez]

Thanks everyone, merging now!