[feature] mermaid.js

[kjellkvinge]

Integrate mermaid into gitea

mermaid claims to "Generate diagrams and flowcharts from text in a similar manner as markdown."

A diagram might be worth more than 1000 words. Even if I see some reasons not to have it, I think this could be a nice feature to have. - So i put this out there and see what happens :)

I have made a simple hack, so this:

```mermaid
graph TD;
    A(stuff)-->B[one];
    A-->C[two];
    A-->D[three];
```

Can look like this:

I have not figured out how to do this whithout toucing source in /vendor.

Heres a sample from a Hugo template:
http://docdock.netlify.com/shortcodes/mermaid/

Regards.

Kjell

[thehowl]

Thank you for your proposal! Unfortunately, I don't think it's a good fit for Gitea.

First of all, you are right saying that a diagram is worth more than a thousand words, and indeed, this would be "handy" in some cases. However, I think those cases can be very easily covered by a simple image (or SVG file) instead of a client-side generated diagram.

Since for many users the use-case for Gitea, and specifically markdown files inside of gitea, is that to document or otherwise describe their projects, almost always these do not involve diagrams; which is why, I think, this would not be suited for Gitea.

[lafriks]

Graphviz?

[sapk]

@kjellkvinge You can use external render via mermaid cli (https://github.com/mermaidjs/mermaid.cli). Implemented in #2570

[kjellkvinge]

Thank you all for reply to this. I guess there is not much interest to a feature like this based on the comments.
As mentioned, I have a working solution for this, so I raised this issue to see if I should crate a PR from this.
I will close this issue for now :)

regards.
kk

[sapk]

@kjellkvinge I use mermaid also for other things but for gitea and more for markdown we try to keep only the common base for everyone and provide easy customization for everyone via external markup render support. This permit to not bloat and limit the code we have to maintain. But feel free to suggest any improvement you can find/code using gitea ;-)

PS: If your solution is only based on JS you can edit file inside custom/templates/custom/header.tmpl to add custom js. In this case we can add your js file under contrib/ or just a link to your solution for mermaid in the documentation for any user that would need it.

[kjellkvinge]

@sapk thank you. I have no problems with the decision to not bloat the markdown feature. It makes sense. I will be happy to contribute later :)

My current solution require change in the vendored blackfriday package. No change in JS, except serving mermaid. - So I would not suggest this as a solution, but if I solve this using JS or the external markdown support, I will put this in the documentation.

thank you.

[rbong]

@sapk I am not sure you can use external renderers for this. Outputting to svg strips the output and attempts to render it as html, outputting to png appears to have no effect.

[rprieto]

I understand that external renderers might make this possible.

I still think it would be a nice built-in addition. It's very common to want to add diagrams to a README, and embedding a PNG does not encourage contributors to keep them up to date. A text-to-diagram syntax is ideal for that.

For what it's worth, GitLab supports it out of the box: https://docs.gitlab.com/ee/user/markdown.html#diagrams-and-flowcharts-using-mermaid (I only bring it up to show it's not an unusual feature to support).

[laoshaw]

all git webui nowadays supports source code well, markdown is now the dominant format for documents, adding mermaid and other markdown features into gitea will be great, inserting svg/png/etc works but not as easily as mermaid

[zeripath]

So I'm actually not against this. The only issue is that we have to be careful not to end up including the world. We really only should load mermaid if we need it too.

I suspect the best way of getting this included in Gitea is to create a feature that requires it - then it's there and we may aswell provide it.

Failing that, it would be genuinely interesting to know how difficult it is for users to add this to their Gitea or other markdown plugins. (For example jsMath)

[laoshaw]

how to submit a request as such? to me gitea is a code repo and markdown repo, the only weaker part is markdown side that could at least support mermaid some some math equations, with mermaid more urgent.

[zeripath]

@laoshaw it's actually really simple to do this. I've just added a docs PR to show how to add Mermaid.js to Gitea

[laoshaw]

Thanks, indeed very helpful. Similar method can be used for MathJax as well, or any other "essential" markdown enhancements. I treat markdown the same important as code in gitea. Github and Gitlab are both providing more extensive markdown supports, hope gitea one day can do the same out of the box.

[zeripath]

It's difficult because we don't want to end up with Gitea being as huge as Gitlab.

I think if we can make things optional in such a way that it's easy to add stuff then everyone wins.

[zeripath]

Regarding MathJax mathematicians and LaTeX users will want to have $$ and $ supported too.

So that needs a Goldmark extension too.

[rprieto]

Thanks @zeripath this is very useful! I personally run Gitea as a Docker image. I would like to avoid creating and maintaining a custom image, with a modified (but up to date) footer.tmpl overlaid.

What would you think of an optional "extension", for example automatically loading Mermaid if a specific ENV variable is set?

[laoshaw]

I think you can put those tmpl files in your Dockerfile (or mapping from host to docker) and just put them under say custom/templates/custom/footer.tmpl, this should do the trick without the need of rebuilding gitea from source.

[rprieto]

You're right that sounds like the easiest way to go. I was more referring to having to check/update the template file and publish a new image when there's a new version of Gitea.

[zeripath]

Take another look at the docs PR I've discovered you can just use custom/footer.tmpl for this.

[laoshaw]

yes I just finish testing it, though I'm not sure what's the difference between footer.tmpl and header.tmpl here, I guess footer.tmpl is better as it will ensure the js lib is loaded before executing anything.

[kenji21]

Take another look at the docs PR I've discovered you can just use custom/footer.tmpl for this.

Thanks for this, but I found an issue when trying to preview mermaid changes in the markdown preview panel (editing a wiki page, or a md file in a repo), it show the mermaid code instead of a graph

But when I manually run:

mermaid.init(".language-mermaid")

the graph is generated in the preview "tab"

I didn't find any "markdown" tmpl file to add this line to (as there is an ajax POST asking to re-render the current edited markdown each time tab changes)

one other solution is to have mermaid listen to the DOM and update the div having language-mermaid class, but I have no idea how to achieve this

[zeripath]

Try adding:

{{if .RequireHighlightJS}}
<script src="https://unpkg.com/[email protected]/dist/mermaid.min.js"></script>
<!-- or wherever you have placed it -->
<script>
  document.addEventListener('DOMContentLoaded', function() {
    mermaid.init(".language-mermaid");
  });
</script>
{{end}}

to custom/footer.tmpl instead?

[kenji21]

didn't worked, but I make some js to do the job (I'm not a js specialist):

{{if .RequireHighlightJS}}
<script src="https://unpkg.com/[email protected]/dist/mermaid.min.js"></script>
<!-- or wherever you have placed it -->
<script>
document.addEventListener('DOMContentLoaded', function() {
    var reinitialiseMermaidHookInstalledOnFirstPreviewClick = false;
    var timerId;
    var debounceFunction = function(func, delay) {
        clearTimeout(timerId)
        timerId = setTimeout(func, delay)
    }

    $('a[data-tab=preview]').each(function() {
        this.addEventListener("click", function(e) {
            if( reinitialiseMermaidHookInstalledOnFirstPreviewClick == false ) {
                /* using div[class=editor-preview] isn't viable with wiki editor (not added yet) */
                $('div[data-tab=preview], div[class="CodeMirror cm-s-paper CodeMirror-wrap"]').each(function(index) {
                    /* https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver */
                    const targetNode = this
                    const config = { attributes: false, childList: true, subtree: true };
                    const callback = function(mutationsList, observer) {
                        for(let mutation of mutationsList) {
                            if (mutation.type === 'childList') {
                                debounceFunction(function() {
                                    /* console.log('mermain reload'); */
                                    mermaid.init(".language-mermaid");
                                    reinitialiseMermaid = false;
                                }, 100);
                            }
                        }
                    };
                    const observer = new MutationObserver(callback);
                    observer.observe(targetNode, config);
                    // Later, you can stop observing
                    //observer.disconnect();
                });
            }
            reinitialiseMermaidHookInstalledOnFirstPreviewClick = true;
        });
    });

    mermaid.init(".language-mermaid");
});
</script>
{{end}}

Took some time to optimize this js to not reinitialise mermaid too many times
Works with both wiki editor and markdown file edition from a repo

[edit] : first solution not working when in footer.tmpl (ok via console), so I edited to make it complete and ready to add to footer.tmpl

[bryanpedini]

embedding a PNG does not encourage contributors to keep them up to date. A text-to-diagram syntax is ideal for that.

@rprieto please take a look at mermaid.cli (and it's output options to generate SVGs from Mermaid markdowns), and, if you know it, maybe drift your eyes a little bit towards it's Docker container implementation too (safely containerized and ready-as-is environment, who needs more than this?)

PS: you can even integrate a Cron job or something similar to replace the image in the uploads folder based on another file (like CHARTS.md in your repo becomes the Mermaid chart in your README once every 6 hours, and every time you push to that CHARTS file, in less than a quarter of a day you have an updated image on your readme; just find . -type f -iname "<seemingly-random-id-of-the-image>.svg" inside Gitea's uploads directory and replace it in a cron bash script, you know...) 😉

[rprieto]

Hi @bryanpedini, that's an option. How would the cron-job work with the uploads folder? Would it run on the same server as Gitea? How would the README files reference these images?

Another option is a pre-commit hook on every repo that runs the Mermaid CLI / container.

But regarding "who needs more than this": we could also could argue "who needs Markdown rendering or Pygment highlighting, just run a CLI and push a README.html page" 🙂 So the question sounds like whether Mermaid is standard enough to make the cut. GitLab decided yes, it seems that Gitea decided no.

[bryanpedini]

Remember that GitLab works on Rails, has like 100 times or somewhere around there more requirements of disk space and computational power, it's noticeably slower for adding seemingly mostly unused features, etc...

I mean, if the Gitea devs decided to remain "slim", that doesn't mean you cannot integrate things in it, like templates and resources customizations and custom integrations.
Regarding the cron job, I was thinking of committing an image in your README, like hosted somewhere else and liked, or uploaded to Gitea's public directory ($custom/public/img), and once integrated in your readme, you can update the underlying image without changing it's name, effectively updating also the markdown file.
A pre-update hook also could do the job very very well and only on-demand, but on large projects it could be very intense to rebuild the images or even just to check if the changed file is related to the charts alone, at every commit. There it's just a matter of deciding which solution works best for your size of environment (you could even have both, using a method for a project and the other for another repo).
Basically you
No matter which you choose (hook or cron), the job is simple: "fetch" a file containing the Mermaid syntax "code", run it through either the globally installed mermaid.cli or the container, and finally replace the hosted image with the current output. There you have it: now the readme is updated too (since it references an external image).

[bryanpedini]

Let me know if there is anything unclear, or if you like the situation, and I'll answer asap accordingly!
Thanks!

[rprieto]

Thank you for all these details. The small disk/memory requirements is one great thing about Gitea. Because Mermaid can be fully rendered client-side, the now-official solution customizing custom/footer.tmpl (#9872) seems ideal.

[silverwind]

The RequireHighlightJS suggestion in the docs is not a good one as we don't even use that library any more and the variable will need to go eventually. I would just hook that initialization on the existance of the .language-mermaid class instead which on 1.13 is server-rendered so that ugly mutation observer stuff above should not be necessary.

[silverwind]

Mermaid integration in #12334.

[ghost]

I had a problem showing mermaid diagrams in the web editor preview in gitea. The problem is, when you run the initialisation of mermaid it only works on existing DOM elements. If you switch between 'write' and 'preview' mode, the DOM elements get deleted and mermaid looses track of it. So you have to reinitialize mermaid every time when you click the preview button and the DOM content is changed.
Following footer.tmpl will archive this functionality:

{{if .RequireHighlightJS}}
<script src="https://unpkg.com/[email protected]/dist/mermaid.min.js"></script>
<!-- or wherever you have placed it -->
<script>
  //initialize mermaid on all existing elements
  mermaid.initialize({flowchart:{useMaxWidth:true}}); 
  mermaid.init(".language-mermaid")

  //initalize an dom observer to dynamically initialize mermaid
  const initMermaidObserver = function () {
    console.log("init mermaid observer")

    const mermaidCallback = function (mutationList) {
      mutationList.filter(function (f) { return f.target instanceof HTMLPreElement && f.addedNodes && f.addedNodes.length > 0; })
        .map(function (mutationRecord) {
          return mutationRecord.addedNodes;
        })
        .forEach(function (addedNodesList) {
          addedNodesList.forEach(function (node) {
            if ((" " + node.className + " ").indexOf(" language-mermaid ") === -1) {
              return;
            }

            mermaid.init(node);
          });
      });
    }

    const mermaidObserver = new MutationObserver(mermaidCallback);
    const mermaidObserveOptions = {
      subtree: true,
      childList: true
    };

    document.querySelectorAll("div.ui.attached.segment")
      .forEach(function (element) {
        mermaidObserver.observe(element, mermaidObserveOptions);
      });
  }

  document.addEventListener('DOMContentLoaded', initMermaidObserver);
</script>
{{end}}