[2.4.1] stricter generic checks causes out-of-memory crash

[emmanueltouzery]

TypeScript Version: 2.4.1
node version: v6.10.3, linux x86-64

Code
https://github.com/emmanueltouzery/ng-typeview

clone the project, run:

• npm install
• tsc

Expected behavior:
the project compiled just fine up to typescript 2.3.

Actual behavior:
with 2.4.1, I get an out of memory at every run:

<--- Last few GCs --->

   44131 ms: Mark-sweep 1338.5 (1416.3) -> 1334.0 (1420.3) MB, 2112.0 / 0.0 ms [allocation failure] [GC in old space requested].
   46537 ms: Mark-sweep 1334.0 (1420.3) -> 1334.0 (1384.3) MB, 2406.1 / 0.0 ms [last resort gc].
   48675 ms: Mark-sweep 1334.0 (1384.3) -> 1334.0 (1384.3) MB, 2137.8 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x38bd260cfb39 <JS Object>
    1: set [native collection.js:~252] [pc=0x1a7da7795be3] (this=0x237efe5f2b39 <a Map with map 0x16e45b60a191>,r=0x3ebe4546afa1 <String[15]: 1359681,1359683>,z=1)
    2: /* anonymous */ [/home/emmanuel/.npm-packages/lib/node_modules/typescript/lib/tsc.js:~977] [pc=0x1a7da7953990] (this=0x163899f5ce41 <JS Global Object>,value=1,key=0x3ebe4546afa1 <String[15]: 1359681,1359683>)
    3: arguments ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0xde09ee [node]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, v8::internal::JSMapIterator, 2>::Allocate(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [node]
 7: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, v8::internal::JSMapIterator, 2>::Rehash(v8::internal::Handle<v8::internal::OrderedHashMap>, int) [node]
 8: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, v8::internal::JSMapIterator, 2>::EnsureGrowable(v8::internal::Handle<v8::internal::OrderedHashMap>) [node]
 9: v8::internal::Runtime_MapGrow(int, v8::internal::Object**, v8::internal::Isolate*) [node]
10: 0x1a7da6c092a7
tsc --version
[1]    5209 abort (core dumped)  tsc

I've seen similar bug reports but no obvious solution (a bug suggested to increase the typescript dependency version, tried to increase that to 2.4.1, didn't help, tried to remove the typedoc dependency which has a transitive dependency to typescript, that didn't help either), and in some of these bugs typescript maintainers ask for the source and reporters couldn't provide it: here I can.

[IrickNcqa]

Same problem - different project. I'm using the angular template from the "Microsoft ASP.NET Core JavaScript Services" @ https://github.com/aspnet/JavaScriptServices, and was getting the exact same error while using Webpack's HMR in Node.

In trying to reproduce the problem from a minimal startup, I may have solved the problem in my case.

It appears that the rxjs/Subject.d.ts file no longer has valid definitions with the improved generic typechecking in v2.4.1. When I try a stock template, I get the following error in the browser:

ERROR in [at-loader] ./node_modules/rxjs/Subject.d.ts:16:22 TS2415: Class 'Subject<T>' incorrectly extends base class 'Observable<T>'. Types of property 'lift' are incompatible. Type '<R>(operator: Operator<T, R>) => Observable<T>' is not assignable to type '<R>(operator: Operator<T, R>) => Observable<R>'. Type 'Observable<T>' is not assignable to type 'Observable<R>'. Type 'T' is not assignable to type 'R'.

Alright, so that leads me to the skipLibCheck flag in the tsconfig.json file. The template includes the skipDefaultLibCheck flag set to true, but doesn't include the skipLibCheck flag. Double checking the flags at https://www.typescriptlang.org/docs/handbook/compiler-options.html, skipDefaultLibCheck is now deprecated in favor of skipLibCheck.

So, I simply replaced skipDefaultLibCheck with skipLibCheck (still set as true). That fixed the browser error about the d.ts file, and it also fixed the heap out of memory error in node (I'm guessing Webpack was silently failing on build because the now invalid d.ts file was being included, which led to the memory issue in node).

I checked your tsconfig.json file, and you don't have the skipLibCheck flag set to true. Maybe give that a shot and see if it fixes your problem too?

[myknbani]

Affects me too when I bumped TS to 2.4.1. Works fine with 2.3.4

[emmanueltouzery]

@IrickNcqa you're correct! Setting up skipLibCheck in my tsconfig.json gives me now normal build errors, which I assume are correct and due to the increased strictness of TS2.4.
My dependencies are quite up to date though. But I guess I must live with the flag for some time if I want to use 2.4.

[myknbani]

I can confirm that setting skipLibCheck to true works.

[myknbani]

Nevertheless, shouldn't this be considered a regression, since we did not need the skipLibCheck option before 2.4.1?

[emmanueltouzery]

@myknbani regardless of anything, tsc should never crash like it does in this bug. at the very least, even if the type definitions are invalid, it should display a proper error message.

That said, from my understanding, skipLibCheck won't be needed anymore once the libraries are updated for typescript 2.4. I do wonder whether we have worst type-safety for the broken libraries compared to TS2.3.x in the meantime though?

[IrickNcqa]

I'd say it's to be expected that stricter type checking would result in some new errors in existing type definition files. Given the above, it's clear rxjs needs to update their d.ts file, and in another thread it looks like the Redux d.ts is going to need an update also. There will probably be a lot of others.

So this will probably boil down to filing issues with the third parties to update their definition files or fixing ourselves and submitting. The major packages will probably be updated fairly quickly. TS v2.4.1 only dropped 20 hours ago - so they definitely haven't had time yet.

A bad d.ts file leading to a out of memory error is bizarre. Agree that needs improvement.

[IrickNcqa]

Looks like the heap error has been around. I just saw it in the v2.4.2 milestone, but no discussion of the skipLibCheck there. https://github.com/Microsoft/TypeScript/issues/16234

[emmanueltouzery]

for my app i'm not even sure which dependencies cause the issue. I suspect maybe immutable but I reproduce also removing that dependency. So a good error message should tell us which dependency is to blame otherwise it's going to be complicated to fix.

what I'd really like to know as well is, what is the downside of skipLibCheck?

does that make the type-checking of the libraries "lazy" and... if you don't use the feature which is badly mapped in the type definition then you have no problem and no regression in type-safety from 2.3. If you do use the feature which is badly mapped, then you still get the crash..

OR does it disable some type-safety and you are worse off on 2.4 with that option compared to staying on 2.3?

but yeah whether it's #1 or #2 it's probably better not enabling the option and sticking to 2.3 until the type definitions are ported to 2.4...

[IrickNcqa]

This bug is percolating all over the place. Just for rxjs...
Rxjs - https://github.com/ReactiveX/rxjs/issues/2539
Thread above: https://github.com/Microsoft/TypeScript/issues/16234
Another thread: https://github.com/Microsoft/TypeScript/issues/16593
Another thread: https://github.com/angular/angular/issues/17800?_pjax=%23js-repo-pjax-container

[sandersn]

I did a git bisect and found that it is a bug in the stricter generic checks introduced after 2.4.0 (at commit 368e5af).

For me, --noStrictGenericChecks also circumvents the crash, without sacrificing any other kinds of type checking. That's the workaround I'd recommend until the bug is fixed.

[sandersn]

By the way @IrickNcqa, #16234 is almost certainly a different issues since it predates the stricter generic type checking. There are lots of ways the Typescript compiler can run out of memory!

[axefrog]

@sandersn Just upgraded from 2.3.x to 2.4.1 and started getting the same error. I can confirm that adding the noStrictGenericChecks flag to my tsconfig successfully circumvents the issue.

[emmanueltouzery]

I see this bug is not in the 2.4.2 milestone. I think it should be, because it hampers libraries becoming compatible to 2.4, since it makes it hard for them to understand what's wrong with their type definitions. It seems like a pretty big regression.

[benjamincombes]

Same problem for me. I tried with version 2.3.4, 2.4.1, notStrictGenericChecks and skipLibCheck, I still get the crash.
Here is the error I get:

[52324:0x104001600]   121844 ms: Mark-sweep 1418.7 (1657.5) -> 1418.7 (1657.5) MB, 399.3 / 0.0 ms  allocation failure scavenge might not succeed
[52324:0x104001600]   122261 ms: Mark-sweep 1418.7 (1657.5) -> 1418.8 (1641.5) MB, 416.5 / 0.0 ms  last resort
[52324:0x104001600]   122698 ms: Mark-sweep 1418.8 (1641.5) -> 1418.8 (1641.5) MB, 435.3 / 0.0 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3a13bc129891 <JS Object>
    1: DoJoin(aka DoJoin) [native array.js:~96] [pc=0x7c3faf560ed](this=0x2c06d9182311 <undefined>,p=0x3dd6a137341 <JS Array[150]>,q=150,E=0x2c06d91823b1 <true>,A=0x2c06d91dc5f1 <String[1]:  >,z=0x2c06d9182421 <false>)
    2: Join(aka Join) [native array.js:~121] [pc=0x7c3fc83aee8](this=0x2c06d9182311 <undefined>,p=0x3dd6a137341 <JS Array[150]>,q=150,A=0x2c06d91dc5f1 <String[1]:  >,z=0x2c06d918...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/usr/local/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/usr/local/bin/node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [/usr/local/bin/node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
 5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
 6: v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
 7: 0x7c3f978437d
Abort trap: 6```

[sandersn]

#17984 fixes the crashes I observed trying to compile ng-typeview and monet. For the others, you can typescript@next starting tomorrow to see whether the fix applies.

[myitcv]

@sandersn - using:

$ tsc --version
Version 2.6.0-dev.2017082

I'm getting an a "heap out of memory" crash:

<--- Last few GCs --->

[92152:0x2d18600]    75250 ms: Mark-sweep 1408.2 (1510.9) -> 1408.1 (1491.9) MB, 3022.1 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 3022 ms) last resort
[92152:0x2d18600]    78273 ms: Mark-sweep 1408.1 (1491.9) -> 1408.1 (1491.9) MB, 3022.4 / 0.0 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x246a2c41bbd9 <JS Object>
    1: set [native collection.js:~247] [pc=0x340ce51f52ba](this=0xeb0a3e61ec9 <a Map with map 0x226b3f014319>,p=0x3c6f900dd919 <String[8]: 889900:2>,x=0x3401e985a811 <a Type with map 0x226b3f0585d1>)
    2: instantiateTypeNoAlias(aka instantiateTypeNoAlias) [/home/myitcv/mgo/src/modelogiq.com/frontend/node_modules/typescript/lib/tsc.js:~24926] [pc=0x340ce58be0ae](this=0x246a2c402241 <undefined>...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x136849c [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, v8::internal::JSMapIterator, 2>::Allocate(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [node]
 7: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, v8::internal::JSMapIterator, 2>::Rehash(v8::internal::Handle<v8::internal::OrderedHashMap>, int) [node]
 8: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, v8::internal::JSMapIterator, 2>::EnsureGrowable(v8::internal::Handle<v8::internal::OrderedHashMap>) [node]
 9: v8::internal::Runtime_MapGrow(int, v8::internal::Object**, v8::internal::Isolate*) [node]
10: 0x340ce50040bd
Aborted (core dumped)

If I set noStrictGenericChecks to true then all seems well. Is this the right "fix"?

[mhegazy]

@myitcv mind filing a new issue and sharing your project

[myitcv]

@mhegazy - happy to file a new issue, but we can't share the code unfortunately.

[mhegazy]

We would be happy to sign any NDA required to get access to the source. without the sources, or a sample reproduction of the issue, i am afraid there is not much we can do.

[sandersn]

@myitcv please do file a new issue to track this. Even if you can't share the code, we are planning to improve the way the compiler fails when the cache grows too large, so you may be able to debug the problem yourself.

In the meantime, --noStrictGenericChecks is the best workaround.

[myitcv]

@sandersn - since #18231 landed (although I haven't narrowed it down to that exact PR, that just happened to be a significant PR that I saw land) the problem has gone away; no --noStrictGenericChecks required. Hence no need to open another issue.

At this point, the only outstanding issue prevent us moving back to the nightly build is something else you're working on, namely immutable-js/immutable-js#1285. I wonder if you or @leebyron can help move that along?

[sandersn]

Immutable is looking for new maintainers right now so I'm not sure @leebyron will have time. One possibility is to decouple the types and put immutable-js's types back on DefinitelyTyped. But that's not a good long-term strategy without agreement from the maintainers of immutable.

[myitcv]

Immutable is looking for new maintainers right now...

That's not the conclusion I took from @leebyron's comments in immutable-js/immutable-js#1215 (comment).

There are of course ways in which I can use your proposed PR changes for now.

But I was hoping given that we're on Typescript 2.5 and that these breaking changes landed in 2.4 that we'd be able to move things along.

[amitm02]

I experience this issue in [email protected] when using [email protected]

[mhegazy]

@amitm02 this is rather an old issue. please file a new ticket, and give us enough context to be able to diagnose the issue.