MDX 3

[kachkaev]

MDX 2 was released on 1 Feb 2022: https://github.com/mdx-js/mdx/releases/tag/2.0.0
UPD: MDX 3 was released on 24 Oct 2023: https://github.com/mdx-js/mdx/releases/tag/3.0.0

The changes are summarised in the blog post: https://mdxjs.com/blog/v2/
UPD: https://mdxjs.com/migrating/v3/

Here is a big one for Prettier:

Originally, the format was very close to how markdown, and HTML in markdown, works. We found that the old behavior often yielded unexpected results. In version 2, we shift the format a little bit closer to how JS(X) works.

The example below is from the blog post. It is a valid MDX in v2, but Prettier 2.5.1 is unable to parse it:

Prettier 2.5.1
Playground link

--parser mdx

Input:

<div>*hi*?</div>

<div>
  # hi?
</div>

<main>
  <div>

    # hi?

  </div>
</main>

Output:

SyntaxError: Unexpected token (2:12)
  1 | <$><main>
> 2 |   <div></$>
    |            ^

Expected behavior:

No syntax error

---

Let’s track MDX 2 3 support here 👀

[sosukesuzuki]

Remark and MDX ecosystems use Pure ECMAScript Modules Packages. We need to use that parser, and I think the our ECMAScript Modules migration(#10157) block this work.

[sosukesuzuki]

For now, As a first step, I will restart the migration to remark-parser v9 (#11938)

[wooorm]

As you’re already bundling prettier when releasing, could you bundle mdx-js/mdx / remark-parse and such as CJS?

[sosukesuzuki]

Do you mean bundling ESM Packages(mdx-js/mdx, remark-parser) into CJS(bundled prettier)? It may be possible, but it will no longer be possible to run the source code as-is in Node.js, as is current Prettier for development.

/cc @fisker

[wooorm]

but it will no longer be possible to run the source code as-is in Node.js

I’m suggesting something like an npm script that works like this:

npx esbuild @mdx-js/mdx --bundle --platform=node --outfile=vendor/mdx.js

That could be run once in a while. Like, every month? It doesn’t have to affect running Prettier in development?

[sosukesuzuki]

Thank you! I think we can use the way you suggested. It may solve the problem that we cannot use other Pure ESM packages I'll try it.

[dburrows]

Just to add, when the parsing does work, the inline markdown within components in MDX2 is mangled - see the Playground example here

Input

<div>
  ## foo
  
  a paragraph
  
  another paragraph
</div>

Output

<div>## foo a paragraph another paragraph</div>

Expected output

<div>
  ## foo

  a paragraph
  
  another paragraph
</div>

Not sure whether the parser update will fix this.

[sebastienbarre]

Thank you people for working on this.
If this helps, here is another problem I'm experiencing in a MDX file which I was just told is related to this very issue:

MDX also supports JavaScript expressions inside curly braces, i.e.: {Math.PI * 2}

Prettier automatically changes it to:

MDX also supports JavaScript expressions inside curly braces, i.e.: {Math.PI \* 2}

Which breaks MDX parsers -- this is no longer a valid Javascript/JSX expression and it won't be executed.

Using Prettier's recommended <!-- prettier-ignore --> won't help because it breaks the MDX parser as well (because of the < used for React/JSX components), and using MDX's recommend {/* this is a comment */} doesn't work because it is not recognized by Prettier since the file is treated as Markdown (as it should), not JSX.

Thank you.

[sosukesuzuki]

we unpin this issue temporary because we should pin #12931

[tats-u]

We have to note that new MDX parsers will not treat * and ** surrounded by both punctuation (including non-ASCII ones) and non-space character as the beginning or end mark of emphasis.
This is due to a regression in CommonMark 0.14+.
commonmark/commonmark-spec#650
This change is inconvenient mostly for Chinese and Japanese documents.

**この部分は強調とみなされなくなりました。**ご注意ください。

[wooorm]

@tats-u I think your time is better spent working on a solution rather than spamming all projects that do something with markdown. If you don’t know how to solve it, you could try and motivate other Chinese or Japanese speakers involved.

[tats-u]

@wooorm I would not like to post an extra comment, but sou should not have used the verb "spam" before evaluating my suggestion on that issue even though I should have posted it on issues or PRs that directly mention on upgrading remark-parse instead.
Your comment this time that chose "spam" as verb looks like you have tried to hinder me from spreading the existence and the bad effect of the eater egg in CM.
I think I have just finished the enlightenment phase in GitHub and do not have to let users of other projects in GitHub know this eater egg.
Prettier users have been relatively sensitive to breaking changes in Markdown written in CJK languages (e.g. #6385).

You must have noted that this change will have to be added in the change log in the version that updates the remark parser and MDX before posting the comment.

[tats-u]

(looks like I mistakenly somehow posted the same comment twice)

[JounQin]

@tats-u That does not matter with this issue at all, your linked issue is caused by upstream, we won't consider that when try to support MDX 2/3. So I would agree with @wooorm that your comments are really spam here, sorry.

[tats-u]

I forgot to say that some test cases related to CJK languages might have to be modified when upgrading remark to newer versions to support MDX v3 (PRs trying to do are stagnant).
This is also why I mentioned that issue here.
I believe this is not a spam begging Prettier not to upgrade remark but just an admonition for those who are trying to upgrade remark and Chinese and Japanese users who want to use the latest MDX even in Prettier. We basically have to accept this eater egg at least for a while because the latest MDX produces more pros for Prettier users.
I will not add any more comments (at least would like to suppress the number of ones) here as this is not so much related and welcomed as you say.

[JounQin]

I won't fix such cases at least, they would be known issues if I'm developing, it is out scope of prettier itself.

[namka279]

MDX #15221

[levino]

So how would one go about this? Update the mdx dependency and fix all the tests? Is it that simple? Is there a WIP branch on this?

[karlhorky]

I think this may be the first PR to watch here, because upgrading unified and remark are required first:

• Upgrade to latest remark version #13676

[levino]

I am toying with the idea to do some coding too. So I would make my own PR or continue someone others work in my own PR. The PR you referenced looks pretty heavy handed to me. Pretty hard to understand what is going on. And all the changes to the snapshots make me wary. Will people really accept these?