fix memory leak in syntax highlighting (causing gray webviews)

rehype-highlight, the syntax highlighting lib we use, has a memory leak bug (remarkjs/react-markdown#791). Downgrading to rehype-highlight@^6.0.0 as recommended fixes the memory leak.

This was causing an issue where the Cody chat webview turned gray due to an out-of-memory situation.

- Fixes #4431
- Fixes https://linear.app/sourcegraph/issue/CODY-2062/gray-screen-in-chat
- Fixes https://linear.app/sourcegraph/issue/CODY-2017/reports-of-chat-ui-lagginess-on-long-messages
- Fixes https://community.sourcegraph.com/t/a-condensed-report-on-key-issues-affecting-cody/439#h-8-performance-issues-with-vscode-extension-18

vscode/CHANGELOG.md

@@ -10,6 +10,7 @@ This is a log of all notable changes to Cody for VS Code. [Unreleased] changes a
 
 - Chat: Don't append @ when "Add context" is pressed multiple times. [pull/4439](https://github.com/sourcegraph/cody/pull/4439)
 - Performance: Reduced the performance overhead for certain types of context fetching, especially for larger files. This might have caused issues with slow autocomplete before. [pull/4446](https://github.com/sourcegraph/cody/pull/4446)
+- Chat: Fixed an issue where the chat view would crash and display a gray screen in VS Code due to an out-of-memory situation. [pull/4459](https://github.com/sourcegraph/cody/pull/4459)
 
 ### Changed
 

vscode/package.json

@@ -1414,7 +1414,7 @@
     "parse-git-diff": "^0.0.14",
     "proxy-agent": "^6.4.0",
     "react-markdown": "^9.0.1",
-    "rehype-highlight": "^7.0.0",
+    "rehype-highlight": "^6.0.0",
     "rehype-sanitize": "^6.0.0",
     "remark-gfm": "^4.0.0",
     "semver": "^7.5.4",

vscode/webviews/components/MarkdownFromCody.tsx

@@ -82,7 +82,7 @@ function markdownPluginProps(): Pick<
     'rehypePlugins' | 'remarkPlugins'
 > {
     if (_markdownPluginProps) {
-        //return _markdownPluginProps
+        return _markdownPluginProps
     }
 
     _markdownPluginProps = {
@@ -102,13 +102,23 @@ function markdownPluginProps(): Pick<
                 } satisfies RehypeSanitizeOptions,
             ],
             [
-                rehypeHighlight,
+                // HACK(sqs): Need to use rehype-highlight@^6.0.0 to avoid a memory leak
+                // (https://github.com/remarkjs/react-markdown/issues/791), but the types are
+                // slightly off.
+                rehypeHighlight as any,
                 {
                     detect: true,
                     languages: Object.fromEntries(
                         Object.entries(all).filter(([language]) => LANGUAGES.includes(language))
                     ),
-                } satisfies RehypeHighlightOptions,
+
+                    // `ignoreMissing: true` is required to avoid errors when trying to highlight
+                    // partial code blocks received from the LLM that have (e.g.) "```p" for
+                    // "```python". This is only needed on rehype-highlight@^6.0.0, which we needed
+                    // to downgrade to in order to avoid a memory leak
+                    // (https://github.com/remarkjs/react-markdown/issues/791).
+                    ignoreMissing: true,
+                } satisfies RehypeHighlightOptions & { ignoreMissing: boolean },
             ],
         ],
         remarkPlugins: [remarkGFM],