feat(core, mdx-loader): deduplicate MDX compilation - `siteConfig.future.experimental_faster.mdxCrossCompilerCache` #10479

slorber · 2024-09-05T17:58:10Z

Docusaurus Faster

This PR is part of the Docusaurus Faster project aiming at reducing production build times

Motivation

When building in production mode, we compile the MDX document twice, for server/client environment.

However, it is kind of useless: in most cases, the MDX loader will be configured the same way for client/server. It's the case for all our content plugins. The only way to have the 2 compilation outputs to diverge with our content plugins would be to use the undocumented vfile.data.compilerName (that I'd like to remove later).

AB Benchmark

Locally modified code for benchmarking + bypass SSG step:

  if (
    options.useCrossCompilerCache &&
    process.env.NODE_ENV === 'production' &&
    process.env.DOCUSAURUS_AB_BENCHMARK === 'true'
  ) {
    options = {
      ...options,
      crossCompilerCache: new Map(),
    };
  }

hyperfine --prepare 'yarn clear:website' --runs 10 'DOCUSAURUS_AB_BENCHMARK=false yarn build:website:fast' 'DOCUSAURUS_AB_BENCHMARK=true yarn build:website:fast'
Benchmark 1: DOCUSAURUS_AB_BENCHMARK=false yarn build:website:fast
  Time (mean ± σ):     37.707 s ±  1.056 s    [User: 61.361 s, System: 9.668 s]
  Range (min … max):   36.611 s … 40.219 s    10 runs

Benchmark 2: DOCUSAURUS_AB_BENCHMARK=true yarn build:website:fast
  Time (mean ± σ):     34.957 s ±  2.804 s    [User: 56.939 s, System: 9.662 s]
  Range (min … max):   32.203 s … 40.326 s    10 runs

Summary
  DOCUSAURUS_AB_BENCHMARK=true yarn build:website:fast ran
    1.08 ± 0.09 times faster than DOCUSAURUS_AB_BENCHMARK=false yarn build:website:fast

Test links

CI + tests, it should work exactly as before

https://deploy-preview-10479--docusaurus-2.netlify.app/

Related issues/PRs

github-actions · 2024-09-05T18:04:13Z

⚡️ Lighthouse report for the deploy preview of this PR

URL	Performance	Accessibility	Best Practices	SEO	Report
/	🟢 91	🟢 98	🟢 96	🟢 100	Report
/docs/installation	🟠 59	🟢 97	🟢 100	🟢 100	Report
/docs/category/getting-started	🟠 74	🟢 100	🟢 100	🟠 86	Report
/blog	🟠 63	🟢 96	🟢 100	🟠 86	Report
/blog/preparing-your-site-for-docusaurus-v3	🔴 48	🟢 92	🟢 100	🟢 100	Report
/blog/tags/release	🟠 64	🟢 96	🟢 100	🟠 86	Report
/blog/tags	🟠 75	🟢 100	🟢 100	🟠 86	Report

github-actions · 2024-09-05T18:04:23Z

Size Change: +333 B (0%)

Total Size: 11.5 MB

Filename	Size	Change
`website/.docusaurus/docusaurus.config.mjs`	27.9 kB	+37 B (+0.13%)

ℹ️ View Unchanged

Filename	Size	Change
`website/.docusaurus/codeTranslations.json`	2 B	0 B
`website/.docusaurus/globalData.json`	30.2 kB	0 B
`website/.docusaurus/i18n.json`	930 B	0 B
`website/.docusaurus/registry.js`	155 kB	0 B
`website/.docusaurus/routes.js`	74.3 kB	0 B
`website/.docusaurus/routesChunkNames.json`	81.7 kB	0 B
`website/.docusaurus/site-metadata.json`	2.17 kB	0 B
`website/build/assets/css/styles.********.css`	113 kB	0 B
`website/build/assets/js/main.********.js`	666 kB	+25 B (0%)
`website/build/blog.html`	61.4 kB	0 B
`website/build/blog/2017/12/14/introducing-docusaurus.html`	74.8 kB	-2 B (0%)
`website/build/blog/2018/04/30/How-I-Converted-Profilo-To-Docusaurus.html`	47.3 kB	+11 B (+0.02%)
`website/build/blog/2018/09/11/Towards-Docusaurus-2.html`	51.7 kB	+11 B (+0.02%)
`website/build/blog/2018/12/14/Happy-First-Birthday-Slash.html`	30.4 kB	-2 B (-0.01%)
`website/build/blog/2019/12/30/docusaurus-2019-recap.html`	39.6 kB	-2 B (-0.01%)
`website/build/blog/2020/01/07/tribute-to-endi.html`	34 kB	-2 B (-0.01%)
`website/build/blog/2021/01/19/docusaurus-2020-recap.html`	48.9 kB	-2 B (0%)
`website/build/blog/2021/03/09/releasing-docusaurus-i18n.html`	45.1 kB	-2 B (0%)
`website/build/blog/2021/05/12/announcing-docusaurus-two-beta.html`	47.5 kB	-2 B (0%)
`website/build/blog/2021/11/21/algolia-docsearch-migration.html`	53 kB	+11 B (+0.02%)
`website/build/blog/2022/01/24/docusaurus-2021-recap.html`	44.5 kB	-2 B (0%)
`website/build/blog/2022/08/01/announcing-docusaurus-2.0.html`	139 kB	-2 B (0%)
`website/build/blog/2022/09/01/docusaurus-2.1.html`	50.9 kB	-2 B (0%)
`website/build/blog/archive.html`	24.7 kB	0 B
`website/build/blog/authors.html`	46.5 kB	0 B
`website/build/blog/authors/j-marcey.html`	65 kB	0 B
`website/build/blog/authors/josh-cena.html`	43 kB	0 B
`website/build/blog/authors/lex-111.html`	50.2 kB	0 B
`website/build/blog/authors/slorber.html`	61.5 kB	0 B
`website/build/blog/authors/slorber/authors/2.html`	65.4 kB	0 B
`website/build/blog/authors/slorber/authors/3.html`	77.7 kB	0 B
`website/build/blog/authors/slorber/authors/4.html`	44.7 kB	0 B
`website/build/blog/authors/yangshun.html`	57.8 kB	0 B
`website/build/blog/authors/zpao.html`	43.5 kB	0 B
`website/build/blog/page/2.html`	66 kB	0 B
`website/build/blog/page/3.html`	71.7 kB	0 B
`website/build/blog/page/4.html`	64.2 kB	0 B
`website/build/blog/page/5.html`	41.9 kB	0 B
`website/build/blog/preparing-your-site-for-docusaurus-v3.html`	138 kB	-2 B (0%)
`website/build/blog/releases/2.2.html`	49.6 kB	-1 B (0%)
`website/build/blog/releases/2.3.html`	60.8 kB	-1 B (0%)
`website/build/blog/releases/2.4.html`	65.3 kB	-1 B (0%)
`website/build/blog/releases/3.0.html`	111 kB	+11 B (+0.01%)
`website/build/blog/releases/3.1.html`	52 kB	0 B
`website/build/blog/releases/3.2.html`	48.8 kB	-1 B (0%)
`website/build/blog/releases/3.3.html`	55.5 kB	-1 B (0%)
`website/build/blog/releases/3.4.html`	55.8 kB	+6 B (+0.01%)
`website/build/blog/releases/3.5.html`	58.1 kB	+9 B (+0.02%)
`website/build/blog/tags.html`	28.6 kB	0 B
`website/build/blog/upgrading-frontend-dependencies-with-confidence-using-visual-regression-testing.html`	128 kB	-1 B (0%)
`website/build/docs.html`	49.4 kB	+8 B (+0.02%)
`website/build/docs/advanced.html`	31.1 kB	-1 B (0%)
`website/build/docs/advanced/architecture.html`	30.7 kB	+3 B (+0.01%)
`website/build/docs/advanced/client.html`	76 kB	-1 B (0%)
`website/build/docs/advanced/plugins.html`	58.6 kB	-1 B (0%)
`website/build/docs/advanced/routing.html`	73.9 kB	+7 B (+0.01%)
`website/build/docs/advanced/ssg.html`	81.5 kB	-1 B (0%)
`website/build/docs/api/docusaurus-config.html`	202 kB	-1 B (0%)
`website/build/docs/api/misc/@docusaurus/eslint-plugin.html`	48.3 kB	+3 B (+0.01%)
`website/build/docs/api/misc/@docusaurus/eslint-plugin/no-html-links.html`	38.1 kB	-1 B (0%)
`website/build/docs/api/misc/@docusaurus/eslint-plugin/no-untranslated-text.html`	37 kB	-1 B (0%)
`website/build/docs/api/misc/@docusaurus/eslint-plugin/prefer-docusaurus-heading.html`	38.3 kB	+3 B (+0.01%)
`website/build/docs/api/misc/@docusaurus/eslint-plugin/string-literal-i18n-messages.html`	41.9 kB	-1 B (0%)
`website/build/docs/api/misc/@docusaurus/logger.html`	39.8 kB	-1 B (0%)
`website/build/docs/api/misc/create-docusaurus.html`	34.9 kB	-1 B (0%)
`website/build/docs/api/plugin-methods.html`	67.4 kB	+3 B (0%)
`website/build/docs/api/plugin-methods/extend-infrastructure.html`	63 kB	+3 B (0%)
`website/build/docs/api/plugin-methods/i18n-lifecycles.html`	60.7 kB	+3 B (0%)
`website/build/docs/api/plugin-methods/lifecycle-apis.html`	170 kB	-1 B (0%)
`website/build/docs/api/plugin-methods/static-methods.html`	46.5 kB	+3 B (+0.01%)
`website/build/docs/api/plugins.html`	31.7 kB	-1 B (0%)
`website/build/docs/api/plugins/@docusaurus/plugin-client-redirects.html`	62.6 kB	+12 B (+0.02%)
`website/build/docs/api/plugins/@docusaurus/plugin-content-blog.html`	198 kB	+12 B (+0.01%)
`website/build/docs/api/plugins/@docusaurus/plugin-content-docs.html`	202 kB	+8 B (0%)
`website/build/docs/api/plugins/@docusaurus/plugin-content-pages.html`	78 kB	-1 B (0%)
`website/build/docs/api/plugins/@docusaurus/plugin-debug.html`	50.5 kB	-1 B (0%)
`website/build/docs/api/plugins/@docusaurus/plugin-google-analytics.html`	51.5 kB	-1 B (0%)
`website/build/docs/api/plugins/@docusaurus/plugin-google-gtag.html`	51 kB	-5 B (-0.01%)
`website/build/docs/api/plugins/@docusaurus/plugin-google-tag-manager.html`	49.8 kB	-1 B (0%)
`website/build/docs/api/plugins/@docusaurus/plugin-ideal-image.html`	51.1 kB	+3 B (+0.01%)
`website/build/docs/api/plugins/@docusaurus/plugin-pwa.html`	122 kB	+3 B (0%)
`website/build/docs/api/plugins/@docusaurus/plugin-sitemap.html`	68.5 kB	+5 B (+0.01%)
`website/build/docs/api/plugins/@docusaurus/plugin-vercel-analytics.html`	42.2 kB	+11 B (+0.03%)
`website/build/docs/api/themes.html`	30.4 kB	-1 B (0%)
`website/build/docs/api/themes/@docusaurus/theme-classic.html`	46.4 kB	+3 B (+0.01%)
`website/build/docs/api/themes/@docusaurus/theme-live-codeblock.html`	38.6 kB	+3 B (+0.01%)
`website/build/docs/api/themes/@docusaurus/theme-mermaid.html`	37.5 kB	+3 B (+0.01%)
`website/build/docs/api/themes/@docusaurus/theme-search-algolia.html`	34.7 kB	-1 B (0%)
`website/build/docs/api/themes/configuration.html`	256 kB	+6 B (0%)
`website/build/docs/blog.html`	219 kB	+5 B (0%)
`website/build/docs/browser-support.html`	51 kB	-1 B (0%)
`website/build/docs/category/getting-started.html`	28.3 kB	0 B
`website/build/docs/category/guides.html`	36.4 kB	0 B
`website/build/docs/cli.html`	63.8 kB	0 B
`website/build/docs/configuration.html`	98.5 kB	+3 B (0%)
`website/build/docs/create-doc.html`	65.3 kB	+11 B (+0.02%)
`website/build/docs/creating-pages.html`	59.1 kB	0 B
`website/build/docs/deployment.html`	211 kB	+8 B (0%)
`website/build/docs/docs-introduction.html`	53.6 kB	-1 B (0%)
`website/build/docs/docs-multi-instance.html`	78.7 kB	+3 B (0%)
`website/build/docs/docusaurus-core.html`	245 kB	-1 B (0%)
`website/build/docs/guides/whats-next.html`	32.8 kB	-1 B (0%)
`website/build/docs/i18n/crowdin.html`	151 kB	+11 B (+0.01%)
`website/build/docs/i18n/git.html`	82.7 kB	+3 B (0%)
`website/build/docs/i18n/introduction.html`	50.8 kB	-1 B (0%)
`website/build/docs/i18n/tutorial.html`	171 kB	+12 B (+0.01%)
`website/build/docs/installation.html`	73 kB	+11 B (+0.02%)
`website/build/docs/introduction/index.html`	280 B	0 B
`website/build/docs/markdown-features.html`	83.2 kB	0 B
`website/build/docs/markdown-features/admonitions.html`	119 kB	+3 B (0%)
`website/build/docs/markdown-features/assets.html`	93.5 kB	+1 B (0%)
`website/build/docs/markdown-features/code-blocks.html`	245 kB	+3 B (0%)
`website/build/docs/markdown-features/diagrams.html`	56 kB	+13 B (+0.02%)
`website/build/docs/markdown-features/head-metadata.html`	52.8 kB	-1 B (0%)
`website/build/docs/markdown-features/links.html`	43.1 kB	-1 B (0%)
`website/build/docs/markdown-features/math-equations.html`	95.4 kB	+10 B (+0.01%)
`website/build/docs/markdown-features/plugins.html`	101 kB	+7 B (+0.01%)
`website/build/docs/markdown-features/react.html`	144 kB	0 B
`website/build/docs/markdown-features/tabs.html`	148 kB	-1 B (0%)
`website/build/docs/markdown-features/toc.html`	88.8 kB	+3 B (0%)
`website/build/docs/migration.html`	43 kB	-2 B (0%)
`website/build/docs/migration/v2.html`	41.3 kB	-1 B (0%)
`website/build/docs/migration/v2/automated.html`	42.6 kB	-1 B (0%)
`website/build/docs/migration/v2/manual.html`	207 kB	-1 B (0%)
`website/build/docs/migration/v2/translated-sites.html`	52.8 kB	-1 B (0%)
`website/build/docs/migration/v2/versioned-sites.html`	67.2 kB	-1 B (0%)
`website/build/docs/migration/v3.html`	218 kB	0 B
`website/build/docs/playground.html`	32.5 kB	-1 B (0%)
`website/build/docs/resources/index.html`	325 B	0 B
`website/build/docs/search.html`	121 kB	0 B
`website/build/docs/seo.html`	93 kB	+12 B (+0.01%)
`website/build/docs/sidebar.html`	136 kB	+3 B (0%)
`website/build/docs/sidebar/autogenerated.html`	156 kB	+12 B (+0.01%)
`website/build/docs/sidebar/items.html`	188 kB	-1 B (0%)
`website/build/docs/sidebar/multiple-sidebars.html`	67.1 kB	+3 B (0%)
`website/build/docs/static-assets.html`	56.3 kB	+3 B (+0.01%)
`website/build/docs/styling-layout.html`	141 kB	-1 B (0%)
`website/build/docs/support/index.html`	319 B	0 B
`website/build/docs/swizzling.html`	120 kB	+13 B (+0.01%)
`website/build/docs/team/index.html`	310 B	0 B
`website/build/docs/typescript-support.html`	65.4 kB	-1 B (0%)
`website/build/docs/using-plugins.html`	115 kB	+12 B (+0.01%)
`website/build/docs/versioning.html`	87.1 kB	+12 B (+0.01%)
`website/build/index.html`	37.8 kB	0 B

_{compressed-size-action}

netlify · 2024-09-06T06:53:10Z

✅ [V2]

Name	Link
🔨 Latest commit	`644a31f`
🔍 Latest deploy log	https://app.netlify.com/sites/docusaurus-2/deploys/66dac46741f5c500080b0f49
😎 Deploy Preview	https://deploy-preview-10479--docusaurus-2.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

slorber added 5 commits September 5, 2024 18:00

simpler mdxLoader code

68058c4

crossCompilerCache

f3c6b15

add faster.mdxCrossCompilerCache option

92c971f

wire mdx cross-compiler cache

95f0dbf

fix imports

59a3255

slorber added the pr: new feature This PR adds a new API or behavior. label Sep 5, 2024

slorber requested a review from Josh-Cena as a code owner September 5, 2024 17:58

facebook-github-bot added the CLA Signed Signed Facebook CLA label Sep 5, 2024

use resource as cache key

91edbd4

slorber added 2 commits September 6, 2024 09:05

use resource as cache key

7a34568

Merge branch 'main' into slorber/mdx-cross-compiler-cache-test

644a31f

slorber merged commit 5bab0b5 into main Sep 6, 2024
36 checks passed

slorber deleted the slorber/mdx-cross-compiler-cache-test branch September 6, 2024 14:07

This was referenced Oct 3, 2024

Image is broken on the main page of docusaurus.io/docs #10544

Closed

fix(mdx-loader): fix cross-compiler cache randomly loading mdx with client/server envs #10553

Merged

Docusaurus Faster #10556

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(core, mdx-loader): deduplicate MDX compilation - `siteConfig.future.experimental_faster.mdxCrossCompilerCache` #10479

feat(core, mdx-loader): deduplicate MDX compilation - `siteConfig.future.experimental_faster.mdxCrossCompilerCache` #10479

slorber commented Sep 5, 2024 •

edited

Loading

github-actions bot commented Sep 5, 2024 •

edited

Loading

github-actions bot commented Sep 5, 2024 •

edited

Loading

netlify bot commented Sep 6, 2024 •

edited

Loading

feat(core, mdx-loader): deduplicate MDX compilation - siteConfig.future.experimental_faster.mdxCrossCompilerCache #10479

feat(core, mdx-loader): deduplicate MDX compilation - siteConfig.future.experimental_faster.mdxCrossCompilerCache #10479

Conversation

slorber commented Sep 5, 2024 • edited Loading

Docusaurus Faster

Motivation

AB Benchmark

Test links

Related issues/PRs

github-actions bot commented Sep 5, 2024 • edited Loading

⚡️ Lighthouse report for the deploy preview of this PR

github-actions bot commented Sep 5, 2024 • edited Loading

netlify bot commented Sep 6, 2024 • edited Loading

✅ [V2]

feat(core, mdx-loader): deduplicate MDX compilation - `siteConfig.future.experimental_faster.mdxCrossCompilerCache` #10479

feat(core, mdx-loader): deduplicate MDX compilation - `siteConfig.future.experimental_faster.mdxCrossCompilerCache` #10479

slorber commented Sep 5, 2024 •

edited

Loading

github-actions bot commented Sep 5, 2024 •

edited

Loading

github-actions bot commented Sep 5, 2024 •

edited

Loading

netlify bot commented Sep 6, 2024 •

edited

Loading