[Website Generator] Links must be .html for nodejs.org/api but they're actually .md on github

[srcmake]

The doc/api folder is full of .md files that serve as the documentation for the Node.js JavaScript libraries. These files are also what's used in the nodejs.org website for the website's api documentation.

Evidently, there's a tool that converts these .md files into the doc folder into the webpages on nodejs.org. (And I think that tool gets run as part of the Release process for each Node.js version bump. I'm not 100% sure about that, though.)

The issue is that URLs in the doc files that link to other doc files must be formatted with a .html extension to work properly for the website. For example, this PR that tries to change .html extensions to .md can't be merged because it would break the nodejs.org api documentation.

This is a problem because on github, the links don't work properly because the file names on github have .md extensions, not .html. Which means that links on github in those files don't work because the file being referenced (ex. doc-name.html) doesn't exist.

The solution would be to update the website api doc generation tool (which I think is this) to convert any .md extensions in URLs to .html when going through the doc files, so that we can update URLs in those docs to be .md so that they link correctly to the actual files on github.

[Trott]

The solution would be to update the website api doc generation tool (which I think is this) to convert any .md extensions in URLs to .html when going through the doc files, so that we can update URLs in those docs to be .md so that they link correctly to the actual files on github.

There's a wrinkle because there are .md links that should be kept as .md links. If it's a link to elsewhere within the API docs, then it should be changed to .html. Otherwise, it should not be changed. For example, in the os.md file, there is a link to our BUILDING.md file. That should not be changed. Similarly, documentation.md contains a link to our CONTRIBUTING.md file that should also not be changed.

[Trott]

/ping @rubys although I don't think they're doing much on the project these days.

[vsemozhetbyt]

It is also more than just extensions. The #hash parts of the links are also different. Compare the GitHub-tools-generated heading anchors in .md and Node.js-tools-generated heading anchors in .html:

https://github.com/nodejs/node/blob/master/doc/api/buffer.md#class-buffer
https://nodejs.org/api/buffer.html#buffer_class_buffer

[grapheo12]

Hello! Is this still open? I would like to work on it.

[srcmake]

@grapheo12 As far as I know, it's still open.

[grapheo12]

So what do I do? Make something that parses each and every .md file in the doc/api folder and convert any .md link to corresponding .html link.
Additionally, I would also maintain an exception list for those files that @Trott mentioned.
Please reply if I am going in the right direction.

[srcmake]

@grapheo12 Well. The idea is to update the existing scripts that generate the Node API documentation website (linked above) to convert any .md links (where appropriate) to be .html. So if the script sees a reference to "async-hooks.md", the script will realize "oh, that file is going to end up being an HTML file for the website. I should just pretend the link is "async-hooks.html" instead.

After that happens, we can change any .html links in the api folder to .md so that they can work on github.

The exact implementation of the first step depends on how the website generation tool works.

[akash-joshi]

Which website generation tool is used ? Would like to pick this issue up

[vsemozhetbyt]

@akash-joshi Scripts that convert .md files into .html files are placed in tools/doc folder. How they are launched, see in Makefile (search for doc or generate.js).

[MarekLabuz]

Hello everyone, I came up with an idea how to solve this issue. I’ve noticed that docs markdown files are created in a way, that links references are located at the end of a file.

synopsis.md

(...)

[Command Line Options]: cli.html#cli_command_line_options
[this guide]: https://nodejs.org/en/download/package-manager/
[web server]: http.html

Instead of creating a seperate file, that maps between *.md and *.html, I suggest transforming markdown files according to the example:

synopsis.md

(...)

[Command Line Options]: cli.md#command_line_options
[this guide]: https://nodejs.org/en/download/package-manager/
[web server]: http.md

[Command Line Options `.html`]: cli.html#cli_command_line_options
[web server `.html`]: http.html

Currently, process that transforms docs from markdown to html goes through several stages. I created a solution that consists in an additional stage applied at the beginning of this process. It replaces links references to their "`.html` version". The replacement is done in memory, so only output html files gain new urls and no content of actual files is changed. Output markdown files remove unused "`.html` versions" of links references. Additionally, the replacement happens only if the "`.html` version" of a link reference exists, it allows for gradual update of documentation without breaking it.