Refactor shikiji syntax highlighting code #9083
Conversation
🦋 Changeset detectedLatest commit: 3b0d7d0 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
github-actions
bot
added
feat: markdown
| const highlighter = await getCachedHighlighter({ | ||
| langs: [lang], | ||
| themes: Object.values(experimentalThemes).length ? Object.values(experimentalThemes) : [theme], | ||
| theme, | ||
| experimentalThemes, | ||
| wrap, | ||
| }); |
There was a problem hiding this comment.
With the abstracted function, we can pass the individual options directly, and it'll handle and process them internally.
There was a problem hiding this comment.
Everything is moved to this file. Code is mostly taken from Code.astro which uses the transforms option to replace certain properties. Compared to other implementations that used regexes.
| }, | ||
| line(node) { | ||
| // Add "user-select: none;" for "+"/"-" diff symbols. | ||
| // Transform `<span class="line"><span style="...">+ something</span></span> | ||
| // into `<span class="line"><span style="..."><span style="user-select: none;">+</span> something</span></span>` | ||
| if (lang === 'diff') { | ||
| const innerSpanNode = node.children[0]; | ||
| const innerSpanTextNode = | ||
| innerSpanNode?.type === 'element' && innerSpanNode.children?.[0]; | ||
|
|
||
| if (innerSpanTextNode && innerSpanTextNode.type === 'text') { | ||
| const start = innerSpanTextNode.value[0]; | ||
| if (start === '+' || start === '-') { | ||
| innerSpanTextNode.value = innerSpanTextNode.value.slice(1); | ||
| innerSpanNode.children.unshift({ | ||
| type: 'element', | ||
| tagName: 'span', | ||
| properties: { style: 'user-select: none;' }, | ||
| children: [{ type: 'text', value: start }], | ||
| }); | ||
| } | ||
| } | ||
| } | ||
| }, |
There was a problem hiding this comment.
Most of the code in this file is copied, except this part which is new. I ported it from the regex version.
astro/packages/markdown/remark/src/remark-shiki.ts
Lines 84 to 90 in 5ef89ef
It's slightly more code now, but with access to the hast, this should be a liitle more performant and robust.
* main: feat(i18n): add `Astro.currentLocale` (withastro#9101) [ci] release (withastro#9107) Add compatibility with cloudflare node (withastro#8925) [ci] format Cancel response stream when connection closes (withastro#9071) [ci] format feat(i18n): apply specific routing logic only to pages (withastro#9091) feat(dev-overlay): Hide plugins into a separate menu when there's too many enabled (withastro#9102) [ci] format Support Svelte 5 (experimental) (withastro#9098) [ci] release (withastro#9078) [ci] format Refactor shikiji syntax highlighting code (withastro#9083) [ci] format fix: Query params trigger the trailingSlash error in preview mode (withastro#9045) fix(assets): bundling regression for specific config on non-Node runtimes (withastro#9087)
Changes
We used to have 3 places that handle shiki/ji syntax highlighting:
Code.astroremark-shiki(markdown and MDX shared)This PR combines all into a single API -
createShikiHighlighter. You can now do something like this instead:I also removed caching for the remark plugin and markdoc plugin for shiki. I don't think they're as useful as these plugins are usually only inited once and reused. Only
Code.astroneeds caching as the user can pass all kinds of configuration.Testing
Existing tests should all pass. And added new test for
createShikiHighlighter.I've also manually tests them, including the error overlay and markdoc, that they work.
Docs
n/a. internal refactoring and shouldn't be visible to users