JavaScript heap out of memory

[Kepro]

/usr/local/bin/node /usr/local/Cellar/yarn/1.9.4/libexec/bin/yarn.js run documentation
yarn run v1.9.4
$ typedoc --out ./documentation ./src --ignoreCompilerErrors

Using TypeScript 2.7.2 from /Users/kepro/myproject/node_modules/typedoc/node_modules/typescript/lib

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
<--- Last few GCs --->
1: 0x100038460 node::Abort() [/usr/local/Cellar/node/10.8.0/bin/node]
2: 0x1000385ea node::FatalTryCatch::~FatalTryCatch() [/usr/local/Cellar/node/10.8.0/bin/node]
3: 0x1001881eb v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node/10.8.0/bin/node]

[12470:0x104002400] 41111 ms: Scavenge 1385.4 (1422.7) -> 1384.7 (1423.2) MB, 3.7 / 0.0 ms (average mu = 0.194, current mu = 0.182) allocation failure
[12470:0x104002400] 41122 ms: Scavenge 1385.5 (1423.2) -> 1384.9 (1424.2) MB, 6.3 / 0.0 ms (average mu = 0.194, current mu = 0.182) allocation failure
[12470:0x104002400] 41131 ms: Scavenge 1385.7 (1424.2) -> 1385.0 (1424.7) MB, 3.1 / 0.0 ms (average mu = 0.194, current mu = 0.182) allocation failure

<--- JS stacktrace --->

==== JS stack trace =========================================

0: ExitFrame [pc: 0xda7ad041bd]
1: StubFrame [pc: 0xda7ad3b51d]

Security context: 0x904b7e1e789
2: getLiteralTypeFromPropertyName(aka getLiteralTypeFromPropertyName) [0x90415c8dae9] [/Users/myproject/node_modules/typedoc/node_modules/typescript/lib/typescript.js:~31511] [pc=0xda7b3de8da](this=0x904454022e1 ,prop=0x90410a97ed1 )
3:...

4: 0x10018818d v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node/10.8.0/bin/node]
5: 0x100461052 v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/Cellar/node/10.8.0/bin/node]
6: 0x100462cf5 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/Cellar/node/10.8.0/bin/node]
7: 0x1004603b7 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/Cellar/node/10.8.0/bin/node]
8: 0x10045f308 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/Cellar/node/10.8.0/bin/node]
9: 0x100467507 v8::internal::Heap::AllocateRawWithRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/Cellar/node/10.8.0/bin/node]
10: 0x100446f64 v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/Cellar/node/10.8.0/bin/node]
11: 0x10052d597 v8::internal::String::SlowFlatten(v8::internal::Handlev8::internal::ConsString, v8::internal::PretenureFlag) [/usr/local/Cellar/node/10.8.0/bin/node]
12: 0x100548670 v8::internal::String::SlowEquals(v8::internal::Handlev8::internal::String, v8::internal::Handlev8::internal::String) [/usr/local/Cellar/node/10.8.0/bin/node]
13: 0x1006844a8 v8::internal::Runtime_StringEqual(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/Cellar/node/10.8.0/bin/node]
14: 0xda7ad041bd
error Command failed with signal "SIGABRT".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Process finished with exit code 1

[purushottamjha]

I am facing this issue too.

[sophistifunk]

I'm getting this on my Mac Pro w/32gb, running on TypeScript itself. Expecting another 32gb to arrive next week, I'll try again then.

[Gerrit0]

Does anyone have a project which causes this behavior published? I'd like to do some debugging to see if I can figure out where the issue is.

[sophistifunk]

Like I said, I got this running in /src in Microsoft's TypeScript repo on master. I'm working on some tooling which consumes TypeDoc's JSON output, and wanted a larger example file than the one I've been using for testing.

[Gerrit0]

Ah, that's what you meant by running on TypeScript. Thanks! I hope to be able to fix this, as soon as I can get strict null checks turned on I'll take a look.

[sophistifunk]

FWIW, just tried with 64gb, still getting node OOM, but it didn't put any pressure on the system, never went below about 20gb free.

[Pauan]

We're getting this as well on our large project. As a workaround, we're doing this:

"scripts": {
    "docs": "cross-env NODE_OPTIONS=--max_old_space_size=8192 typedoc [...]"
}

It uses the cross-env package.

[Gerrit0]

Much later than I'd have liked, but I started taking a look at this today. There may have been an unintentional (partial?) fix, as TypeDoc handled the TypeScript repo just fine, never going over 1.6gb ram. This still seems high to me, so I'm going to keep looking at it.

What I've done:

git clone https://github.com/TypeStrong/typedoc.git
cd typedoc
npm i
npm run build
git clone https://github.com/Microsoft/TypeScript.git
cd TypeScript
npm i
npm run build
node ../typedoc/bin/typedoc src --json typedoc.json --tsconfig src/tsconfig.json --ignoreCompilerErrors

This resulted in a 1298043 line json file.

---

If anyone has a project where TypeDoc still runs out of memory from the master branch, it should help me track down any remaining issues.

[Gerrit0]

Closing as this seems to be fixed, please open a new issue if you run into this problem again.

[adidahiya]

I had similar OOM issues after upgrading to typedoc 0.15.0-0 which I resolved by downgrading to 0.14.2. I think this issue should be re-opened.

<--- Last few GCs --->

[24128:0x103945000]    23521 ms: Mark-sweep 1392.0 (1424.3) -> 1391.5 (1424.3) MB, 974.0 / 0.0 ms  (average mu = 0.079, current mu = 0.005) allocation failure scavenge might not succeed
[24128:0x103945000]    24918 ms: Mark-sweep 1392.4 (1424.3) -> 1391.9 (1424.8) MB, 1392.2 / 0.0 ms  (average mu = 0.034, current mu = 0.003) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x144f6fedbe3d]
    1: StubFrame [pc: 0x144f6fedd1ff]
Security context: 0x318aeab9e6e1 <JSObject>
    2: getTypeListId(aka getTypeListId) [0x318a62b856b9] [/Volumes/git/oss/blueprint/node_modules/typedoc/node_modules/typescript/lib/typescript.js:~38660] [pc=0x144f707fe92c](this=0x318ac26026f1 <undefined>,types=0x318a1c527f79 <JSArray[2187]>)
    3: getUnionTypeFromSortedList(aka getUnionTypeFromSortedList) [0x318a62b...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x1000387af node::Abort() [/usr/local/Cellar/node@10/10.15.2/bin/node]
 2: 0x100038980 node::FatalTryCatch::~FatalTryCatch() [/usr/local/Cellar/node@10/10.15.2/bin/node]
 3: 0x10016eee3 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node@10/10.15.2/bin/node]
 4: 0x10016ee84 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node@10/10.15.2/bin/node]
 5: 0x10041fdf4 v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/Cellar/node@10/10.15.2/bin/node]
 6: 0x1004219c3 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/Cellar/node@10/10.15.2/bin/node]
 7: 0x10041f0ff v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/Cellar/node@10/10.15.2/bin/node]
 8: 0x10041df4c v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/Cellar/node@10/10.15.2/bin/node]
 9: 0x10042642f v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/Cellar/node@10/10.15.2/bin/node]
10: 0x10042673c v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/Cellar/node@10/10.15.2/bin/node]
11: 0x100406cef v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/Cellar/node@10/10.15.2/bin/node]
12: 0x1005e000e v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/Cellar/node@10/10.15.2/bin/node]
13: 0x144f6fedbe3d
14: 0x144f6fedd1ff

You can see the full build logs here, but the build setup has a few layers to it because we use typedoc through documentalist. I will try to come back with more details to pinpoint the issue. This is a large TS project, but I still had the issue when I restricted typedoc to running on only one of the packages in the monorepo instead of all of them.

[Gerrit0]

@adidahiya could you open a new issue with this? I suspect it is a different issue from the original cause as the inadvertent fix in 0.14.something worked.

[millsp]

Hi @Gerrit0, I'm also getting the same error on my repo

https://github.com/pirix-gh/ts-toolbelt

npm run build:docs

This happens since my last update, where I made type distribution a default. So I suspect that typedoc tries to unfold types too much. I'd be nice if it could preserve the type "as it is" in the code.