MDX v2: ☂️ Umbrella issue

[johno]

👋 Hey folks, the time has come to begin putting together MDX v2. We've been aggregating feedback, working on parsing, and we think we now have all of our 🦆🦆🦆🦆 ducks in a line to begin releasing a next tag in the next week.

Please note: Nothing written here is final and we'd love for contributors, users, and anyone else to comment with ideas, opinions, and questions/concerns.

Features

• 🤖Drastically improved parsing led by @wooorm in Rewrite how MDX is parsed #1039. This will handle a wide array of parsing issues that have been brought up over the last year with a robust solution. The new parser will address a large amount of papercuts users have been working around for far too long. It also adds JSX usage into the AST which will allow users to perform their own transforms with plugins. One of the more notable additions here is the ability to support expressions throughout a document (also a breaking change).
• 🚀Official Vue support led by @codebender828 and @ChristopherBiscardi in Vue, rebased #1029. The previous alpha implementation of Vue was buggy and not updated with MDX as it evolved. We no longer have a custom transpiler for Vue and only have to implement a custom renderer/pragma. This is so huge and we're delighted @codebender828 took on this monumental effort.
• 🧩TypeScript support led by @ChristianMurphy. The public facing packages feature official types with tests.

Breaking changes?

Generally speaking, most of the breaking changes will be internal and shouldn't directly affect end users. We haven't yet shipped them, though, because they are technically changing behavior users may've been depending on (which will be documented below).

However, with the parsing overhaul (#1039) the syntax has been formalized which will likely cause issues for characters that aren't HTML encoded, < and {, (some of which we'll be able to address with codemods).

• Parsing changes outlined in Rewrite how MDX is parsed #1039
• components for the pragma prop has been renamed to mdxComponents Remove components prop merging in MDXCreateElement #638
• Removal of layoutProps being passed to MDXContent/wrapper Remove layoutProps from transpiled output #742
• Remove deprecated mdPlugins and hastPlugins options Remove option support for mdPlugins and hastPlugins #718

RFCs

The following RFCs (request for comments) are either accepted (✅), being discussed (💬), declined (✖️), or need to be written (📝). If you have opinions here, please chime in on these issues as they will affect the v2 release.

• ✅ Officially drop node 8 support RFC: Officially drop node 8 support and remove it from testing in v2 #978
• ✖️Consolidate pre/code/inlineCode RFC: Consolidate pre/code/inlineCode #743
• ✅ MDX plugins (this will likely land as 2.2) RFC: MDX plugins #741
• ✅ Interleaving Markdown in JSX (PR Rewrite how MDX is parsed #1039) RFC: Interleaving Markdown in JSX #628
• 💬 MDX comment syntax RFC: MDX comment syntax #1042
• ✅ MDXs (this will likely land as 2.3, and as a plugin) RFC: Multiple MDXs in a Single File #454
• ✅ let/const/var support (this will likely land as 2.1) RFC: Support variable declaration (const/let/var) #1046
• ✅ Remove autolink syntax RFC: Remove autolink support ✅ #1049
• 💬 MDX block parsing RFC: MDX v2 block parsing #1170

Work left to complete or currently in flight

If you have any interest in working on any of these tasks please let us know! We'd be happy to help guide anyone along.

• Update eslint-mdx Support for MDX v2 eslint-mdx#204
• Sync with upstream/downstream dependencies Sync with upstream/downstream projects regarding v2 #1045
• Implement the new homepage and website graciously designed by egghead 🎬 MDX landing page #444
• Vue documentation Vue documentation #851
• Implement v1 and v2 benchmarks Add benchmarking for MDX parsing and compiling performance #1044
• Implement v1 to v2 codemods for MDX files Implement a codemod for syntax changes we can detect #1043
• Update the necessary TypeScript types Add TypeScript types to the next branch #1078
• Use babel standalone instead of core Explore moving to babel standalone or offering a more browser friendly build #704
• Add ability to opt out of literal HTML elements using MDXProvider Don't replace literal HTML elements with MDXProvider #821
• Improve import/export parsing
  • Improve export name extraction Improve export name extraction for shortcode generation #1160
  • export { default } from './foo' Make export { default } from "…" work #961
  • import * as Foo from './foo' Add support for namespace imports #1050
• Missing MDXProvider docs Document MDXProvider composition #919
• Only call makeShortcodes when there are shortcodes being used If there are no shortcodes, don't add makeShortcode function to output JSX #721
• Document an MDX spec and an MDXJS spec that aren't tied to unified
• MDX showcase 📸 Built with MDX showcase #414
• Handle JSX member expressions Handle JSX member expressions #953
• Start publishing canaries under the next tag
• Take v2 canaries on a spin with open source sites, ask for the community to test on their sites
• Consolidate remark-mdx and remark-mdxjs, imports and exports can be turned off with an option
• Create an mdx package that includes all packages with exports like: mdx/react and mdx/vue

Encountered bugs

• Curlies in JS strings v2 parser can't handle the open curly in JS strings #1081
• Serializing inline links Reserializing of inline links back to MDX result in improper JSX #1169

Performance considerations

• Dropping down to one babel step, perhaps returning JS rather than JSX by default MDX performance #1152

How can you help?

Let us know if you want to take on any of the issues that don't yet have folks assigned. You can also take the canary for a spin and let us know if you encounter any problems:

yarn add @mdx-js/mdx@next @mdx-js/react@next

---

Is there something missing? Is there something you're concerned about?

[chasemccoy]

This is so exciting to see! When the time is right, I’d be happy to help test this with our design system: https://seeds.sproutsocial.com

[codebender828]

Thanks @johno ! Excited for this to progress. I guess the first thing we can do for the Vue package is to work on the documentation so we can get some people to use it and gather some feedback. Then when it's stable we can see how to normalize the MDX rendering props and attrs for Vue in mdx v2.

[johno]

For sure @codebender828! Are you interested in owning that (#851)?

[codebender828]

@johno
Yes! Sounds good! Where's the best place to get started with familiarizing myself with the docs?

[timedtext]

Is it possible to solve vscode's IntelliSense problem? mdx-js/mdx-analyzer#56

[shilman]

@johno worst question ever, but any updated ETA on this? we've added a bit of scope to Storybook 6 and are looking at RC late june and final release late july. would be amazing to make MDX2 part of that. and--since it's a breaking change--pretty bad if we can't, since we try to limit our major release cycle to once per year (tho we're not religious about that)

storybookjs/storybook#9311

[johno]

Hey @shilman, we're hoping for an RC in the by June, and if all goes well a mid July release. Generally speaking, the breaking changes should be pretty light because they're generally internals aside from the new parsing changes (which we will also offer a codemod that should address most problems that might arise).

Hopefully this corresponds nicely with you folks? I'm pretty busy this week with other things, but plan on picking MDX v2 back up full time starting next week.

[jsamr]

I think I've spotted an edge-case issue with mixed html / md (@mdx-js/[email protected]). See for example this:

<abbr title="Foo Bar">FB</abbr>, I like [links](https://domain.org)

will render as:

<p>
  <abbr title="Foo Bar">FB</abbr>, I like [links](https://domain.org)
</p>

instead of

<p>
  <abbr title="Foo Bar">FB</abbr>, I like <a href="https://domain.org">links</a>
</p>

The workaround I've found is to add a zero-width space at the beginning of the line, like so:

&ZeroWidthSpace;<abbr title="Foo Bar">FB</abbr>, I like [links](https://domain.org)

[axe312ger]

Is there any way to "simply" test it with my Gatsby websites? I'd love to give it a try and report back bugs :)

[axe312ger]

A new version was released. Progress. LOVELY!!!

[axe312ger]

Is there any way to "simply" test it with my Gatsby websites? I'd love to give it a try and report back bugs :)

@wooorm Hey! You just released the RC-1. That is awesome! Can I support you? Is there a way to try it out with Gatsby? :)

Thank you so much!!!

[wooorm]

Hey! Thanks for the kind words. You can indeed support MDX and the underlying projects (and me). To help out financially, see Sponsor on the site. A big part of helping is also just helping out in repos, as in, contributing. See Contribute on the website.

Unfortunately Gatsby integrates heavily with the internals of MDX. That allows Gatsby to do some cool things, and has allowed them to apply other improvements specific to Gatsby, but it means you can’t just switch to the RC of V2 in Gatsby.

It is possible. But hard. @thecodingwizard made it work in cpinitiative/usaco-guide: https://github.com/cpinitiative/usaco-guide. The relevant discussion is here: wooorm/xdm#40. If anyone tries this, swap xdm out with @mdx-js/* — they’re almost identical.

I am assuming that Gatsby will work on adding support for Version 2 themselves though! :)

[axe312ger]

@wooorm Alright, thank you. I contribute to Gatsby itself, so I'll see if I can help over there.

[axe312ger]

I'll consider financial support as soon the rebranding of my tiny agency is done :)

[theoludwig]

Can we have an estimate of the v2 release date?
If there is anything blocking, needs to be done or how can we help to move things forward?

[wooorm]

Hey all!

V2 was finally released, thanks for everyone who contributed, and for everyone who waited! ✨ More on v2 in the release and on the site: https://github.com/mdx-js/mdx/releases/tag/2.0.0.

[kachkaev]

That is amazing! 🎉 🎉 🎉

I have just created an issue in Prettier to discuss MDX 2 support: prettier/prettier#12209

Feel free to contribute with your thoughts 🙌 (or code 🙏 )