Transfer ownership of the Node.js API Docs Build/Publish/Tooling to the Web Infra Team

[ovflowd]

I'd like to formally request that the ownership of https://github.com/nodejs/node/tree/main/tools/doc and https://github.com/nodejs/node/tree/main/doc (all files and subfolders except the api folder, which contains the API docs) be transferred to the Web Infra team.

The reasoning behind this is that it makes sense for the Web Infra team to own the tooling (generation, HTML templates, JavaScript code, CSS styles, etc), build (generation, output, compilation, dependencies), and publish (nodejs.org domain publishing, upload, etc) to the Web Infra team.

We aim to own and maintain these pieces of code and software. We have cool plans for the future of the API docs.

Note: This has nothing to do with the API metadata proposal (from Next-10) since that is a proposal of the change and restructuring of the API docs themselves (the actual Markdown files, and their folder structure)

With the change of ownership for the tooling we aim to simply apply new modern tooling and redesign the API docs to look like the Node.js Website, not intent of changing the structure of the Markdown (the name of the files, amount of output files in HTML and source Markdown would remain the same)

[ovflowd]

cc @nodejs/tsc @nodejs/build

The build team historically owned the publishing process of the API docs (and maybe the build process or at least the part where it gets offloaded to the VMs/DigitalOcean) so we want to ensure they also agree with this transfer of ownership.

[Trott]

Is this a change in CODEOWNERS and an understanding within the project of who "owns" the doc and doc tooling? Or is there more to it than that?

[GeoffreyBooth]

https://github.com/nodejs/node/tree/main/doc is a little too broad, as it contains stuff like https://github.com/nodejs/node/tree/main/doc/contributing which contains policy docs and the like. Maybe scope the folders to the ones that contain the tooling?

[Trott]

https://github.com/nodejs/node/tree/main/doc is a little too broad, as it contains stuff like https://github.com/nodejs/node/tree/main/doc/contributing which contains policy docs and the like. Maybe scope the folders to the ones that contain the tooling?

Or doc/api, which are the docs that end up on the website?

[ovflowd]

I mentioned above in parenthesis that we would only want the ownership of the api_assets and the other api docs-related web assets of that folder 😅

I didnt mention file by file because that could change. But yes, we dont want ownership of api and contributing folders definitely

[ovflowd]

Is this a change in CODEOWNERS and an understanding within the project of who "owns" the doc and doc tooling? Or is there more to it than that?

Something like that. Just who is responsible for maintaining that piece of code.

[Trott]

Will our CODEOWNERS/github-bot tooling work if doc/api is owned by the website team but doc/api/http.md is owned by the http team and so on?

[ovflowd]

Will our CODEOWNERS/github-bot tooling work if doc/api is owned by the website team but doc/api/http.md is owned by the http team and so on?

Reminder we don't want ownership of the doc/api folder, just the doc/ folder or the current files (api_assets, template.html, etc etc)

[Trott]

Oh, yeah, +1 to ownership of doc/api_assets and doc/template.html. If it's just a CODEOWNERS change, I think you can open that as a PR immediately.

[ovflowd]

Right, that part is the easy part. The more complicated part is that we (web infra team) would like to have ownership in the future of the publish of the api docs; instead of the build team. We would be taking that burden from them. (Referring to the nodejs.org/api subdomain and possibly the nodejs.org/docs)

[Trott]

Right, that part is the easy part. The more complicated part is that we (web infra team) would like to have ownership in the future of the publish of the api docs; instead of the build team. We would be taking that burden from them. (Referring to the nodejs.org/api subdomain and possibly the nodejs.org/docs)

I guess I'm not sure what that looks like. (And maybe you don't either beyond the paragraph you wrote about it at the end of the issue description.) Right now (or until very recently), if I recall correctly and understand correctly, docs are built using make docs on Jenkins during a release, and then the resulting files are transferred to the web server. Is your proposal that this would no longer be the case and the web team will build the docs using services that are available generally like GitHub Actions or maybe within the hosting service we're using?

Personally I think that's fine as long as we have a commitment that running make docs within Node.js source code will always produce output identical to what is on the web pages. (And if that's not the case, how much of a difference is tolerable?) Building docs is almost certainly an incidental thing to the build team and I would not be surprised if they never give it much thought or attention until something goes wrong. Handing it off to an already-up-and-running team that will give it a lot of attention is probably good for everyone.

[richardlau]

FWIW It's not a secret (but perhaps less known) but doc generation and upload is in the main Makefile -- start with the doc-upload target. They're currently treated the same as binaries (see the other *-upload Makefile targets).

[ovflowd]

I guess I'm not sure what that looks like. (And maybe you don't either beyond the paragraph you wrote about it at the end of the issue description.) Right now (or until very recently), if I recall correctly and understand correctly, docs are built using make docs on Jenkins during a release, and then the resulting files are transferred to the web server. Is your proposal that this would no longer be the case and the web team will build the docs using services that are available generally like GitHub Actions or maybe within the hosting service we're using?

Something like that. Except probably not being at Vercel, but still Cloudflare.

Personally I think that's fine as long as we have a commitment that running make docs within Node.js source code will always produce output identical to what is on the web pages. (And if that's not the case, how much of a difference is tolerable?) Building docs is almost certainly an incidental thing to the build team and I would not be surprised if they never give it much thought or attention until something goes wrong. Handing it off to an already-up-and-running team that will give it a lot of attention is probably good for everyone.

Right, the makefile generation of docs would still be the same, the published API docs will (this will not change) match and be the same output as from the Makefile command. And Node.js would still bundle the API docs on its distribution.

What would change is only the publishing process of the API docs to the web from the current format, which is deeply bound to the published versions of Node.js, etc.

The idea is to decouple this process, which would allow us to more efficiently manage the update of older API docs and a more straightforward process to upload and publish docs in general.

[aduh95]

The idea is to decouple this process, which would allow us to more efficiently manage the update of older API docs and a more straightforward process to upload and publish docs in general.

Do we have an example when that would have been useful? Coupling docs with releases seems rather useful.

[ovflowd]

The idea is to decouple this process, which would allow us to more efficiently manage the update of older API docs and a more straightforward process to upload and publish docs in general.

Do we have an example when that would have been useful? Coupling docs with releases seems rather useful.

Sorry, let me rephrase, the docs would still be coupled to releases. But we could alter parts of the UI, components, templates and other pieces (such as banners and alerts and search content, and revised navigation) without the need of new Node.js releases

[ovflowd]

In other words a new doc would be generated for the v21 lineup of Node.js, and every new minor release of that lineup = new version of the docs. But updating the actual pages (not the content) wouldn't require a new version of Node.js or alteration of the source of Node.js (if that makes sense)

We want to decouple the actual templates and layouts and assets of the Node.js API doc generation from the node core (such as the ui components and etc)

And allow the docs to be regenerated and etc. This is just an idea but trying to still figure out future paths that would allow us to seamlessly update doc related stuff without the need of touching core or making a minor release or push to a tag/branch

[Trott]

Yeah, being able to update the format (as opposed to content) of the old docs would be a win. There's a longstanding bug wherein the version switcher has some 404 links for older releases and is missing links to newer versions when you're browsing older versions. Attempts have been made to fix it, but unless something has changed since I was paying more attention to Node.js, it always proved to be more difficult to fix than it appears at first blush.

So if this allows us to fix navigation problems in old versions of docs, update color contrast easily for a11y, and so on, then great.

The documentation text of old versions should be frozen except in extreme situations.

[ovflowd]

Yeah, being able to update the format (as opposed to content) of the old docs would be a win. There's a longstanding bug wherein the version switcher has some 404 links for older releases and is missing links to newer versions when you're browsing older versions. Attempts have been made to fix it, but unless something has changed since I was paying more attention to Node.js, it always proved to be more difficult to fix than it appears at first blush.

So if this allows us to fix navigation problems in old versions of docs, update color contrast easily for a11y, and so on, then great.

The documentation text of old versions should be frozen except in extreme situations.

Right, unfortunately with the new tooling the earliest Node.js version that would be supported is v16; that's because before v16 Node.js API docs had multiple files where the YAML snippets within the Markdown were not standard. Ie misformatted yaml blocks, and such.

If there are reasons to revamp any of the older versions than v16 we need to understand the why's and research the effort needed to accommodate such versions.

To be fair the new tooling is probably not needed at all for this revisioned publish process and we could keep the older version of the docs with the older tooling.

The idea here is that we add a GitHub Action Workflow:

• Every new PR generates a preview branch of the API docs on Vercel (build done directly at the action job and then uploaded to Vercel
• Every new commit on main updates the latest version of living API docs published on https://api.nodejs.org (and then we use cloudflare to proxy requests from nodejs.org/api to that domain without actually redirecting them)
  • Alternatively commits on main go to R2 bucket of /docs/ subfolder/latest/
• We can run manual workflows for version branches which then generates and uploads it to R2 (/docs/ bucket) cc @flakey5
  • These pretty much serve to update older versions or specific versions of Node.js docs.

[flakey5]

Just for clarification on how the new docs will be served,

• Every new PR generates a preview branch of the API docs on Vercel (build done directly at the action job and then uploaded to Vercel

Will the docs for a release be served from Vercel or is it just the previews?

• Every new commit on main updates the latest version of living API docs published on https://api.nodejs.org (and then we use cloudflare to proxy requests from nodejs.org/api to that domain without actually redirecting them)

  • Alternatively commits on main go to R2 bucket of /docs/ subfolder/latest/

• We can run manual workflows for version branches which then generates and uploads it to R2 (/docs/ bucket)

  • These pretty much serve to update older versions or specific versions of Node.js docs.

The current setup for serving docs through the release worker is by rewriting any nodejs/docs/... paths to nodejs/release/vX.X.X/docs/ with a sort of virtual directory map (code). With the current setup, the path nodejs/docs shouldn't actually exist in the bucket (I think it might from when it was originally synced).

Would these new docs change that behavior? I.e. nodejs.org/docs/vX.X.X against the worker will actually fetch from nodejs/docs/vX.X.X in the bucket.