Scalability concerns (high memory usage, long build times)

[krillboi]

So I just finished deploying a new version of our docs (https://support.touchgfx.com/) which introduces versioning. It functions well but I noticed some concerning values when it comes to build times and memory usage. This particular version uses docusaurus alpha.54, so if these issues have been relieved in newer versions, I apologise (we will upgrade to newest Docuaurus version before our next release).

First of all, we have to use the --max_old_space_size=16000 option on the node command to get it to finish, else it runs out of memory. It also took about 26 minutes to build the documentation as can be seen in the following image:

Furthermore, I saw upwards of 10gb of ram being used at some points during the build.

For reference, this is building two versions of our docs (each with about 360 .mdx files) + the next version which we don't use (it seems I can disable building a next version in a newer Docusaurus version which is nice).

So it seems that adding versions is really tough for the build times and resources, so I thought I'd bring some awareness to this issue if you hadn't already considered it 😄

Really appreciate the great work you guys are doing!

[slorber]

Hi,

How does it compare to the situation, you had before?

--max_old_space_size=16000 is 16Gb that seems huge.

Have you tried lower values?

Have you tried with another CI, or building locally?

360 docs is already a large site, and everytime you'll cut a new version you'll increase the build time. Unfortunately we can't do much to avoid making the build time increase in such case. If this becomes a problem you can take older versions and publish them as standalone sites.

At some point if your site has many docs, many versions, many languages, yes you will have large build times without splitting the site in smaller subsites.

[krillboi]

The build is currently only being run locally.

I just tried removing one version (which means it's building 1 version + next) and building. Memory usage didn't seem to change much. This is the result:

I also tried building while removing versioning. Memory usage dropped to a max of about 5gb. Result:

I realise that our site is large and that it will of course be unavoidable that build times will increase with every added version. I guess I was just surprised with how much each version seemed to increase build times/resource usage and there might be some optimization that could be done.

But yeah, I guess at some point having subsites for versions/languages could be a good idea.

[slorber]

I'm surprised too that it's so expensive to build your site, as it seems much slower than the Docusaurus 2 website (much less docs, but more versions, and build relatively fast)

If you give me the sources I could check.

Maybe you use too much the ideal image plugin? Image processing without caching has always been a bottleneck (including in Gatsby)

[krillboi]

Is the ideal image plugin something special? We usually use images with our own React components that uses a normal <img> tag.

Looking at our static/img folder, we currently have 544 images taking up 130mb of space.

I'll look into maybe providing you with the sources for the build.

[slorber]

The ideal image is not special, it's just an image transformation plugin, which is indeed a heavy process, that adds more build time everytime you add new ideal images

[krillboi]

@slorber I've gotten the go-ahead to provide you with the sources as long as I can send it to you personally. How should I send it to you?

[krillboi]

I think we're going to go for a multi-domain setup for each language in the future though, but just the versions alone take 50+ min to build.

[slorber]

@RDIL is working on the Webpack 5 migration here and it looks to be working: #4089

Trying to finish a few things before and catch up with some issues, but it is a high priority for me to do this upgrade and see how to improve the build time with some better caching.

Also want to provide a guide on how to scale a Docusaurus site, sharding it by locale, version or docs plugin instance. I think this is already possible but not too idiomatic.

[RDIL]

Hello yes I have arrived

The migration is almost ready, and should help out.

[slorber]

btw already contacted @sserrata on Discord as your site will be a nice case-study to run some benchmarks and we definitively want to improve your build times. Will tell you when it's a good time to work together on finding a solution

[sserrata]

FYI @glicht

[liaocyintl]

just try

yarn build --no-minify

which reduce my building time from 30+m to 3m.

[slorber]

Surprising to see such a huge impact, are you sure it's not Webpack cache kicking in on second build (in addition to disabled modification)?

[liaocyintl]

are you sure it's not Webpack cache kicking in on second build

I did yarn clear before yarn build --no-minify.
It works!

[slorber]

on our own website and my M1 computer --no-minify leads to 50s -> 45s, not as significant, so still surprised 😅 is your site opensource?

[liaocyintl]

is your site opensource?

Unfortunately, it is an internal site.

leads to 50s -> 45s

Maybe the 50s is not a significant long build time. If you generate a lot of MDXes on your site and increase the standard building time to more than 30m, --no-minify may be as significant.

[anaclumos]

    ~/Downloads/www  on   main !7 ?1  hyperfine --runs 20 --warmup 3 'npx docusaurus clear && npx docusaurus build --no-minify' 'npx docusaurus clear && npx docusaurus build'
Benchmark 1: npx docusaurus clear && npx docusaurus build --no-minify
  Time (mean ± σ):     337.329 s ±  2.618 s    [User: 501.452 s, System: 68.826 s]
  Range (min … max):   332.143 s … 342.655 s    20 runs

Benchmark 2: npx docusaurus clear && npx docusaurus build
  Time (mean ± σ):     430.421 s ±  3.707 s    [User: 880.611 s, System: 83.959 s]
  Range (min … max):   425.927 s … 438.454 s    20 runs

Summary
  'npx docusaurus clear && npx docusaurus build --no-minify' ran
    1.28 ± 0.01 times faster than 'npx docusaurus clear && npx docusaurus build'

✔  took 4h 57m 8s   at 03:00:01 PM 

I confirm that this makes the building 1.3x faster 😄

[OrKoN]

I think we encountered the same problem. We recently moved to Docusarus for Puppeteer and we try to generate a versioned documentation website. At first, the performance seemed good enough but as soon we started publishing more versions the process started to take more memory and time, to the point, when building the site using GitHub Actions fails with out-of-memory errors or takes many hours (not sure if it's hanging or if actually building the site). I wonder if there are workarounds to this problem? The source of the site is here: https://github.com/puppeteer/puppeteer/tree/main/website

[liaocyintl]

GitHub Actions fails with out-of-memory errors

You can try to config the swap of Github Runner as follows.
It works for us.
ref. https://github.com/marketplace/actions/set-swap-space

- name: Set Swap Space
  uses: pierotofy/set-swap-space@master
  with:
    swap-size-gb: 10

[slorber]

@OrKoN once a version becomes unmaintained, I suggest "archiving it" so it's not anymore inside your production SPA and does not increase the build time.

We do this on the Docusaurus site itself:

• https://github.com/facebook/docusaurus/blob/main/website/versionsArchived.json
• https://github.com/facebook/docusaurus/blob/main/website/docusaurus.config.js

It's not an std feature but can be achieved in userland with a dynamic config.

You can also try to use swc (we use it on our website) or esbuild, see #4765

[OrKoN]

Thanks for the tips! We started to "archive" the old versions although it does not seem to be easy with GitHub pages so we just link to the markdown documentation for now. It'd be great if docusaurus could support light-weight versioning too when the previous version could be just moved to a sub-folder without re-building.

[slorber]

Yes @OrKoN , I'd like to provide a docusaurus version:archive 1.0.0 CLI feature or something similar. It is a bit opinionated and possible to achieve in userland already today. I'd rather have more userland success stories of archiving documentation versions before designing this as a first-class feature: we want to implement the opinionated patterns that already work best for our user base.

[madelson]

I think we have a similar issue. We are trying to use Docusaurus to generate an internal site with both hand-written documentation files and auto-generated markdown files for API reference (something like the MSFT docs site).

We have ~3100 markdown files in total, most of which are the auto-generated API reference docs.

With this configuration, production build is very slow (20+ minutes) and often runs out of memory.

[LunaticMuch]

I have a similar problem. Takes 60m on average.

[madelson]

@slorber / @RDIL any thoughts on next steps here?

• Is the cause of the slowness known? Any suggestions on where to look in the docusaurus source?
• Are the maintainers interested in/planning to support larger sites with this tool?

[RDIL]

That's fine. All I really need is package.json, package-lock.json, and your docusaurus config.

[Josh-Cena]

@madelson Remember to output CJS during server build (dynamic imports need to be compiled). Here's our config, before we migrated to SWC:

docusaurus/website/docusaurus.config.js

Lines 95 to 103 in 74f653d

	webpack: {
	jsLoader: (isServer) => ({
	loader: require.resolve('esbuild-loader'),
	options: {
	loader: 'tsx',
	target: isServer ? 'node12' : 'es2017',
	},
	}),
	},

It's node12, not node14. There's a very complicated reason behind this.

[madelson]

@Josh-Cena ok node14->node12 seemed to do the trick. I'd be very curious to understand the "very complicated reason" :-)

Here's what I have now:

    webpack: {
      jsLoader: (isServer) => ({
        loader: require.resolve('esbuild-loader'),
        options: {
          loader: 'tsx',
          format: isServer ? 'cjs' : undefined, // TODO do I need this?
          target: isServer ? 'node12' : 'es2017',
        },
      }),
    },

Any thoughts on whether being explicit about cjs is needed and if so why?

As far as performance, overall this is now clocking in at around 8m overall:

• Client: 1.68m
• Server: 3.34m
• docusaurus-lunr-search <1 min
• "Generated static files in "build": ~2 min
  I'm curious what all is happening in the last step and whether there is anything that can be done to optimize that, but still getting it down to 8 minutes from 22 minutes is a big win. I also haven't tried removing terser yet.

[RDIL]

I do believe that generating static files in build is either Terser doing its thing or a docusaurus post-processing task like link checking.

[RDIL]

Why it has to be Node 12 ^^

[LunaticMuch]

This morning on my CICD.

[2022-09-28T07:56:45.804Z] [INFO] [en] Creating an optimized production build...
[2022-09-28T08:35:40.264Z] [info] [webpackbar] Compiling Client
[2022-09-28T08:35:40.264Z] [info] [webpackbar] Compiling Server
[2022-09-28T08:37:47.266Z] [success] [webpackbar] Client: Compiled successfully in 2.35m
[2022-09-28T08:38:26.602Z] [success] [webpackbar] Server: Compiled successfully in 2.94m
[2022-09-28T08:38:53.114Z] [SUCCESS] Generated static files in "server/public".

Is there a way to get more details on steps executed during the build and their time?

[anaclumos]

More ways to speed up the build? i.e. build multiple locales in parallel?

[slorber]

This should be possible yes, but depends on your infra and is a bit out of the scope of Docusaurus:

• With GitHub CI you could use a GitHub Actions Matrix
• With Netlify/Vercel/Others you could create one separate deployment per language. Using one subdomain per language, or assembling them under a single domain with rewrite rules.