Broken link to tutorial on https://contributing.shields.io/index.html

[bochap]

Are you experiencing an issue with...

• [shields.io] https://contributing.shields.io/index.html

🪲 Description
The link for tutorial on how to add a badge. appears to be broken on the site deployed to contributing.shields.io. The link is pointing to https://contributing.shields.io/doc/TUTORIAL.md which does not exists. The source on https://github.com/badges/shields/blob/master/README.md works though and it links to https://github.com/badges/shields/blob/master/doc/TUTORIAL.md. It looks like the same content is deployed at https://contributing.shields.io/tutorial-tutorial

💡 Possible Solution

[chris48s]

Cheers for raising this. Looking at it, there are actually several places where we've got relative links that do work when you view the docs as a pile of markdown files on Github but then don't work on https://contributing.shields.io/

It would be nice to maintain the property that you can read the docs in either environemnt and the links will all work but I don't have an immediate solution to that.

[chris48s]

I've had a bit of an investigate to see if we can achieve this:

JSDoc doesn't give us any way to configure how the tutorials are linked or intercept that process: https://jsdoc.app/about-tutorials.html

I had a look to see if we could use something like markdown-folder-to-html to pre-process the markdown into HTML before we pass it to JSDoc to use as a tutorial (we can use a custom template to get round doubling up the menus). markdown-folder-to-html does intercept all the markdown links and change them to HTML links, but then JSDoc changes all the HTML filenames which then breaks the links. Again, this is non-configurable.

We can chuck the HTML files into the JSDoc output dir as static files but this feature is only really useful for JS, CSS, images etc because if you use it for HTML, JSDoc doesn't add the files to the menus and that is also non-configurable.

The way to get JSDoc to link all the tutorials up correctly is to link them using {@link} and {@tutorial} tags (example: https://github.com/SoftwareBrothers/admin-bro-dev/blob/master/docs-src/tutorials/05-actions.md ). Doing this gets JSDoc to link all your tutorials up nicely once you compile them, but then you lose the internal linking when you view them on GitHub as markdown.

Potentially we could try and write something bespoke to pre-process the markdown or HTML to find and replace broken internal links, but I think its just more hassle than we need for what it actually achieves. I think the way forward here is to either decide that:

• The tutorials should be viewed as markdown files on GitHub, instead of https://contributing.shields.io/ and just link back to https://github.com/badges/shields/tree/master/doc for the tutorials.
• The tutorials should be compiled and viewed on https://contributing.shields.io/ convert the internal links to JSDoc tags (e.g: replacing [setup instructions](./TUTORIAL.md) with [setup instructions]{@tutorial TUTORIAL} and so on), optimise for that use case and point users there.

Inevitably there are tradeoffs. One of the things we gain if we go all-in on JSDoc is the ability to link the API docs and tutorials up more closely (more useful now we have more of the API documented), but one of the things we lose is anchor links, which is a bit annoying. Simultaneously, we have very few of them.

In either case, we probably also want to replace the repository README with a dedicated front-page for contributing.shields.io as the repo README inherently needs to be optimised for GitHub. I don't mind which of those options we pick and I'm happy to make the necessary changes for either approach, but I think it is useful to get some feedback on the options at this point. While the opinions of core team are important, I'd also particularly value feedback from people who are newer to the project as these are the people who use tutorials and documentation most frequently.

[calebcartwright]

Thanks for digging into this one @chris48s and laying everything out.

With that second option, what, if anything, changes when authoring/updating our documentation markdown files? We have markdown-to-other-markdown file references today, for example the Tutorial has links to the Contributing.md file and doc/service-tests.md, what would those link references look like inside the Tutorial file in the jsdoc all-in scenario? Would those change to be absolute references to the contributing site (https://contributing.shields.io/doc/service-tests.md) or keep the relative service-tests.md link?

[chris48s]

No probs. Its probably much more clear to work through an example than explain it in the abstract.

At the moment, we've got this:

There is a dedicated
[tutorial for tests in the service-tests folder](service-tests.md).
Please follow it to include tests on your pull-request.

which works if you view it as a markdown file on GitHub but then that compiles to:

<p>There is a dedicated
<a href="service-tests.md">tutorial for tests in the service-tests folder</a>.
Please follow it to include tests on your pull-request.</p>

That generates a broken link on https://contributing.shields.io because https://contributing.shields.io/service-tests.md doesn't exist.

---

We could change that to

There is a dedicated
[tutorial for tests in the service-tests folder]{@tutorial service-tests}.
Please follow it to include tests on your pull-request.

which would then compile to:

<p>There is a dedicated
<a href="tutorial-service-tests.html">tutorial for tests in the service-tests folder</a>.
Please follow it to include tests on your pull-request.</p>

That generates a valid link on https://contributing.shields.io because https://contributing.shields.io/tutorial-service-tests.html does exist, but then if you're viewing it as a markdown file on GitHub, {@tutorial service-tests} doesn't link to anything.

[calebcartwright]

Got it thanks. So these truly are mutually exclusive options. Our docs are going to either be navigable within markdown (rendered on GitHub) or navigable on https://contributing.shields.io 👍

[paulmelnikow]

I like the idea of making https://contributing.shields.io the preferred place for the tutorials, and for any documentation that we don't think is a better fit at https://shields.io/. 👍

I'm a little wary of JSDoc's limitations when it comes to the URLs, as it'd be nice to have a clean URL like https://contributing.shields.io/tutorial, though it's probably not worth wrestling with the tool to make that happen.

I know you spent a lot of time evaluating different docs to MD/HTML tools in #3645 and don't want to churn on that, though I wonder if there's another tool that would provide a little more flexibility. Another one we could try is gatsby-source-jsdoc which would let us keep the tooling the same across the website and the doc site. This could definitely be deferred, although it would be kind of nice to settle this first, since we'd have to update the tutorial links again. I'm willing to take that plugin for a spin if that seems helpful.

[chris48s]

That wasn't one of the tools I looked at. I've not tried it out, but just having a look over their parser code, my gut instinct is its going to support a relatively modest subset of JSDoc. If you give it bash, first things I'd check for to "fail fast" would be:

• Does it support @typedef, @module, @abstract, @see, @property?
• Do functions with async, static, get, etc get ignored?

[paulmelnikow]

Yikes, I didn't realize that package had a homegrown parser. It seems like it would make a lot more sense to leverage an existing parser. jsdoc-to-markdown is a possibility; seems like that could be wired up with gatsby-source-filesystem.

However, the Gatsby site itself uses gatsby-transformer-documentationjs. There's a rendered example here.

It's based on documentation.js, which is an entirely different JSDoc parser.

[chris48s]

I didn't look at documentation.js when I was evaluating options, but that does look like it is worth evaluating to see if it could solve a problem for us here.

[paulmelnikow]

Okay, I'll dig in a bit further then.

[chris48s]

I've been meaning to come back to this for quite a while and never quite got round to it, but it turns out today's the day.

I've had a bit of a look at documentation.js and gatsby-transformer-documentationjs.

• documentation.js on its own doesn't really help us - it can parse JSDoc comments and turn them into HTML/markdown/etc, but doesn't incorporate external resources.
• gatsby-transformer-documentationjs can build a docs site mixing generated docs with external resources, but broadly I think we're going to end up with the same issue we have right now which is that the links are going to work in the generated docs site or on github, but not both. For example, gatsby's documentation which is generated using this module has this exact issue, as noted in their README: https://github.com/gatsbyjs/gatsby/tree/master/docs

I don't think either of those libraries solve the problem raised in this issue. This is also a common situation for other static site generators.