🚀 RELEASE: 0.17.0

[chrisjsewell]

I think this is just about good to go!

I have essentially been running a "feedback loop" with executablebooks/MyST-NB#380, to ensure everything will be fine running these changes up to myst-nb and jupyter-book (see jupyter-book/jupyter-book#1608)

My only slight "concerns" are:

1. With ♻️ REFACTOR: Parsing logic of Markdown links #467, a few people might complain about some subtle changes to link identification (e.g. that links with # are no longer "automatically" treated as external), although it should be very rare
2. With ‼️ BREAKING: Remove dollarmath from default myst_enable_extensions #505, again people might not notice, and then complain it is not working

I think both these changes are good/necessary, it is just the backward dependency consideration (although I would hope people have suitably pinned to ~=0.16.0)

---

Breaking changes

Dollarmath is now disabled by default.

You may enable it manually by adding dollarmath to myst_enable_extensions.
If your documentation had syntax like this in it:

$$
e=mc^2
$$

You should ensure that dollarmath is explicitly added to myst_enable_extensions.
See the math extension documentation for more details. by @chrisjsewell in #505

ref: #505

Link parsing has been changed

The MyST Parser will now try to more sensibly parse links that are specified in markdown, with the goal of behaving as users would expect the link to behave. For example, linking to local non-Sphinx files will now cause those local files to be included with your Sphinx build.

This might slightly change the behavior of some markdown links, so double-check your links if you were doing anything non-standard. For example, if you were using link syntax like [link text](link/to/localfile.txt) you should now see more sensible behavior.

See the markdown links/referencing documentation for more details.

ref: #467

What's Changed

• 👌 IMPROVE: Add bullet attribute to bullet_list node by @chrisjsewell in 👌 IMPROVE: Add bullet attribute to bullet_list node #465
• 👌 IMPROVE: Use correct renderer for state.inline_text by @chrisjsewell in 👌 IMPROVE: Use correct renderer for state.inline_text #466
• 🧪 TESTS: Use publish_doctree to test error reporting by @chrisjsewell in 🧪 TESTS: Use publish_doctree to test error reporting #472
• 🧪 TESTS: Ensure myst-docutils does not install sphinx by @chrisjsewell in 🧪 TESTS: Ensure myst-docutils does not install sphinx #473
• ♻️ REFACTOR: default_parser -> create_md_parser by @chrisjsewell in ♻️ REFACTOR: default_parser -> create_md_parser #474
• ⬆️ UPGRADE: Drop python 3.6, support 3.10 by @chrisjsewell in ⬆️ UPGRADE: Drop python 3.6, support 3.10 #475
• 👌 IMPROVE: Docutils parser settings by @chrisjsewell in 👌 IMPROVE: Docutils parser settings #476
• 🐛 FIX: front-matter rendering with docutils by @chrisjsewell in 🐛 FIX: front-matter rendering with docutils #477
• 👌 IMPROVE: Code block highlighting by @chrisjsewell in 👌 IMPROVE: Code block highlighting #478
• 🐛 FIX: Front matter -> sphinx metadata regression by @chrisjsewell in 🐛 FIX: Front matter -> sphinx metadata regression #480
• 👌 IMPROVE: note_refname for docutils internal links by @chrisjsewell in 👌 IMPROVE: note_refname for docutils internal links #481
• 🐛 FIX: Ordered list starting number by @chrisjsewell in 🐛 FIX: Ordered list starting number #483
• 👌 IMPROVE: Propagate enumerated list suffix by @chrisjsewell in 👌 IMPROVE: Propagate enumerated list suffix #484
• 👌 IMPROVE: DocutilsRenderer.create_highlighted_code_block by @chrisjsewell in 👌 IMPROVE: DocutilsRenderer.create_highlighted_code_block #488
• 🐛 FIX: Source line reporting for nested parsing by @chrisjsewell in 🐛 FIX: Source line reporting for nested parsing #490
• 🧪 TESTS: Use pytest-param-files by @chrisjsewell in 🧪 TESTS: Use pytest-param-files #496
• 👌 IMPROVE: Convert nested headings to rubrics by @chrisjsewell in 👌 IMPROVE: Convert nested headings to rubrics #497
• 🔀 MERGE: Internal improvements to the code base by @chrisjsewell in 🔀 MERGE: Internal improvements to the code base #498
• ✨ NEW: Add myst_title_to_header configuration by @chrisjsewell in ✨ NEW: Add myst_title_to_header configuration #492
• 🐛 FIX: linkify link rendering by @chrisjsewell in 🐛 FIX: linkify link rendering #499
• 🧪 TESTS: Update pytest-param-files by @chrisjsewell in 🧪 TESTS: Update pytest-param-files #500
• ✨ NEW: Add myst_gfm_only configuration by @chrisjsewell in ✨ NEW: Add myst_gfm_only configuration #501
• ✨ NEW: Add strikethrough extension by @chrisjsewell in ✨ NEW: Add strikethrough extension #502
• 🔧 MAINTAIN: Implement MockInliner.parse by @chrisjsewell in 🔧 MAINTAIN: Implement MockInliner.parse #504

Full Changelog: v0.16.1...master

[codecov]

Codecov Report

Merging #507 (efc8152) into master (7654462) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #507   +/-   ##
=======================================
  Coverage   90.06%   90.06%           
=======================================
  Files          16       16           
  Lines        2103     2103           
=======================================
  Hits         1894     1894           
  Misses        209      209           

Flag	Coverage Δ
pytests	90.06% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
myst_parser/__init__.py	88.57% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7654462...efc8152. Read the comment docs.

[chrisjsewell]

Any initial comments @choldgraf @mmcky ?

[mmcky]

@chrisjsewell with the breaking change

• ‼️ BREAKING: Remove dollarmath from default myst_enable_extensions #505

just two quick questions:

1. Is there a warning if $$ is "parsed" to tell users of myst-parser to enable the dollarmath extension?
2. would it be possible to show in the CHANGELOG what the output looks like if a user tries to use $$ with the new version of myst-parser and a note to say the dollarmath extension needs to be turned on.

In the past when there was breaking changes on another project they had that in the CHANGELOG and it was immediately identificable as "oh that's my issue, my output looks like that". I presume it just shows the $$ and the raw math markup?

[mmcky]

thanks @chrisjsewell this is looking like a great update.

[befeleme]

Hello, when do you plan to release 0.17? I'd like to update it in Fedora as it will work nicely with the new Pygments :) Thank you!

[chrisjsewell]

Hello, when do you plan to release 0.17? I'd like to update it in Fedora as it will work nicely with the new Pygments :) Thank you!

this week 😄

Just wanted to give time for anyone to raise issues/objections, and double-check the back-breaking changes aren't going to incur too much backlash 😅

[choldgraf]

I've added a short section with "breaking changes" to the top comment above to try and explain these two a bit more. Also added a short example per @mmcky's suggestion. Feel free to take or leave as you wish @chrisjsewell - I am +1 on a release 👍

[mmcky]

thanks @chrisjsewell on this refactor work. I think it looks great.

This month I would like to do a parse of the developer documentation -- it would be great to fully understand the architecture )(so might arrange a zoom with you) so we can identify / document some ways forward in the future to restore cross-document glue behaviour such as:

1. use sphinx transforms/post-transforms to resolve (with limited support for various mime types)

I will move that comment to myst-nb as I think it mainly lives at the myst-nb level. I think that is likely to be the biggest "risk" point for that release.

I am +1 on release as well.

[choldgraf]

Is there anything blocking this release?

[chrisjsewell]

Ok added changelog, so will merge when tests pass and release

I presume it just shows the $$ and the raw math markup?

@mmcky, no not for HTML output, since mathjax (enabled by default in sphinx) will still parse/render them, which is the point of possible confusion.
IMO this is not a good solution, since it is not output format agnostic; the parser (i.e. markdown-it) should be the one to parse the syntax, then something like mathjax or katex, should simply render it (i.e. convert to HTML), which is what the dollarmath extension enables.
In fact, the ideal solution would be for there to be a mathjax/katex implementation in Python, so we could convert the latex math to html (or other formats) during the sphinx build, similar to what happens with code blocks syntax highlighting via https://pygments.org/. Then you would not need any unnecessary JavaScript running on your page load

[mmcky]

Thanks for explaining that for me @chrisjsewell that makes a long of sense. It would also stop that load time when you initially load a large page it sometimes blinks as the mathjax gets rendered.

[choldgraf]

Sent out a tweet about this as well: https://twitter.com/ExecutableBooks/status/1493312075011088388

[chrisjsewell]

Cheers @choldgraf
Note, one more thing may warrant retroactively highlighting in the changelog, is that e.g. [text](#anchor) will no longer work automatically, and instead you should set myst_heading_anchors (see #519)