Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.26.0 increases RAM usage significantly #2607

Closed
yoursunny opened this issue Jun 22, 2024 · 2 comments
Closed

v0.26.0 increases RAM usage significantly #2607

yoursunny opened this issue Jun 22, 2024 · 2 comments
Labels
bug Functionality does not match expectation question Question about functionality
Milestone

Comments

@yoursunny
Copy link

yoursunny commented Jun 22, 2024

I have a large monorepo https://github.com/yoursunny/NDNts that uses TypeDoc.

TypeDoc build steps in my monorepo
corepack pnpm install
corepack pnpm build
corepack pnpm typedoc

When using TypeDoc 0.25.13, the Node process running TypeDoc can run to completion using around 1.5 GB RAM.
After upgrading to TypeDoc 0.26.0, the Node process running TypeDoc needed to consume more than 7.3 GB RAM.
My system is Ubuntu 22.04, Node 20.12.2, TypeScript 5.4; I observed RAM usage with htop command.

The significant RAM increase would cause difficulty in running such builds on GitHub Actions free tier, which has a RAM limit of 7.5 GB.
Is there some way to reduce RAM usage, or break the build into multiple stages and then combine them?

@yoursunny yoursunny added the question Question about functionality label Jun 22, 2024
@Gerrit0 Gerrit0 added the bug Functionality does not match expectation label Jun 22, 2024
@Gerrit0
Copy link
Collaborator

Gerrit0 commented Jun 22, 2024

Part of this is I apparently just forgot how packages mode works, and violated one of the requirements for it.

TypeDoc 0.26 stores a files registry which is used to track what paths belong to... and I extended that with the ability to hold reflections, so relative paths to files would create links... the way I did that was by saving a Reflection object reference, which is terrible because that has effectively doubled the amount of memory TypeDoc needs, even though those references are dead everywhere else.

    // The combination of these two make up the registry
    protected mediaToReflection = new Map<number, Reflection>();
    protected mediaToPath = new Map<number, string>();

    protected reflectionToPath = new Map<Reflection, string>();
    protected pathToMedia = new Map<string, number>();

It's entirely possible there are other issues here too... still looking into it.


It is possible to to run TypeDoc individually on each project, and then merge the results together, the typedoc-packages-example repo has an example of how to do this.

@Gerrit0
Copy link
Collaborator

Gerrit0 commented Jun 22, 2024

Seems like that was the only problem.

0.25
0 25 perf

0.26.1 (soon)
node_128587

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Functionality does not match expectation question Question about functionality
Projects
None yet
Development

No branches or pull requests

2 participants