Differences in HTML rendering between MyST, Myst-NB, JupyterBook, Quarto

[liquidcarbon]

Hello,

First off, I'm really enjoying MyST markdown and the Jupyter Lab extension!

I'm investigating various ways for publishing notebooks; I've been using Quarto for about a year for work and personal projects. Now I decided to learn more about MyST by perusing the docs and Youtube videos from @rowanc1, and I'm a bit confused by the current relationship between all the MyST products. Some of them haven't had releases for a while, some are under active development.

1. Could you please offer some guidance on what's happening?

2. What would you say are the biggest differentiators between MyST and Quarto?

3. A few observations / silent questions:

• seems like https://mystmd.org/, with myst CLI, is the latest and greatest
• there's built-in publishing to GHP and Curvenote - cool!
• the default renderer uses https://github.com/myst-templates/book-theme/ , where the secret sauce is an 11 MB build/index.js file (what does it do?)
• rendering the minimal example:

📦 Installed web libraries in 2.17 s
📖 Built README.md in 79 ms.
📖 Built 01-paper.md in 78 ms.
📖 Built 02-notebook.ipynb in 71 ms.
⚠️  _build/site/public/citations-a9c4703bdc918569eb6ef25547c035e1.png Could not convert from image _build/site/public/citations-a9c4703bdc918569eb6ef25547c035e1.png to webp:
cwebp is required

(more webp warnings follow)

📚 Built 3 pages for project in 396 ms.

• the process generates over 4000 files... ??? most of them small JS files. Can they be bundled together into the HTML? (Quarto has embed-resources option)

[mystmd-quickstart]$ tree . | wc -l
4175

[mystmd-quickstart]$ tree . | grep js | wc -l
3025

[riziles]

@liquidcarbon , just a civilian here who happens to love MyST and the Executable Books project in general. I'm sure someone official will chime in and explain everything, but this is my understanding:

JupyterBook (which I think was the foundation of the project) was built on Sphinx and is awesome if you're a Pythonista and are already familiar with it or need to document Python code. It's also great if you have a bunch of notebooks that might need to run Python code at build time.

MyST, though, is in theory useful to anybody who wants to create some well-formatted documentation. I've used it for things that didn't have any code in them whatsoever. With that in mind, creating a dependency on Python is somewhat unnecessary if the user doesn't need Python. The myst cli allows you to build a LaTeX doc for a scientific journal, a website that can be easily deployed to Curvenote (or any other hosting service for that matter) or just a beautiful well formatted PDF.

MyST and Quarto are both awesome. I'm not completely clear on why they both exist. I wish the syntaxes were more closely aligned, but I won't complain about free stuff.

The myst cli uses lots of cutting edge tools like Node, Unified and Vitest to build a website. The book-theme you referenced above is built on Remix which is built on top of React and esbuild. That's why the output has a ton of JS and very little HTML. Personally, I'm not a fan of React, but in theory you could use the MyST parser (or write your own using the MyST specification) to build an output using a good Javascript framework like Svelte! (Yeah, I said it. Come at me, Zuckerberg.)

[liquidcarbon]

Thanks! I found some insights of yours from before. Very helpful here, too: #385

Silence from Quarto/Posit on Joel's request for comparison...

I found an example of a book rendered with both Quarto and JupyterBook, but it doesn't appear they used any MyST: https://github.com/pnavaro/big-data

[riziles]

Yeah, unfortunately, the syntaxes are different enough that you're probably better off just sticking to CommonMark if you want it to work seamlessly in both.

[rowanc1]

I think @riziles has done another great job of summing up the ecosystem and current efforts, thank you! The many-many files needed for a site are just how the react/remix community works at the moment (there are probably ways to tune this with recent changes to vite/remix, but I haven't looked into it). The important parts of the output are the json files and static assets. When hosting as an application rather than a rendered static site, those assets are the only things that need to be stored, and it allows the theme to be maintained independently of the content. This is more how the scientific publishing world works, and is a pretty big distinction between myst/quarto on how the architecture hangs together. This also allows for exploration of different themes (e.g. that use svelte or plain html) and that only expect these static files to create their build (which would bring things a bit closer to how quarto works with inlined assets).

For the syntax differences, I hope we will take some steps towards bringing things closer together in the coming months. Our goal as a MyST community is that it should be relatively easy to bring your content between these various tools, and really focus on the standards for wider science/technical communication.

[liquidcarbon]

Thanks to both of you @rowanc1 @riziles !

The way I would use the bundle is to do aws s3 sync onto my s3-compatible bucket. I'd put the source MDs in Git, but not the rendered pages. I prefer not to use Github pages, and I don't think 4000 files belong in source control anyway. Seems like the bulk of them are one time assets for navigation, UI and all that, rather than 4000 files per post?

Hope you guys find a way towards a common standard! 😉

[liquidcarbon]

Is there news on OXA link? Such a cool idea!

[rowanc1]

Near and dear to my heart. :) The concepts are what is driving the myst spec/ast and a lot of the thinking around how references are resolved in myst. Hopefully this is coming through in the cross-references pages and external references. The sphinx references are an early start down that path.

Some of the work and demos that curvenote is working on in the scientific publishing industry is to actually allow this to resolve to any existing published figure/equation/section (if published open access, with certain existing standards (JATS)). Which would mean actually pasting in a DOI, and typing @doi:10.123/ABX#fig1 in myst (or something like that) and getting either a link to -- or directly embedding that content with attribution/licensing attached.

Basically the dream behind oxa.link is to have a "package-management-like infrastructure" for science communication in the same way that these exist for software.

[liquidcarbon]

That would be amazing. I wonder why aren't you promoting it more, or at least mentioning it on the Curvenote website? I had to stare at the GIF here for a couple of min to spot the OXA part :) https://curvenote.com/for/jupyter