Allocation failed - JavaScript heap out of memory

[danikane]

Software version

OS: Windows 10 x64
Node: v8.9.4
NPM: v5.6.0
Any other software related to your bug:

What did you get as the error?

I just started a new Quasar project only and saw afterwards Quasar got an upgrade to 0.16.
Consistently getting the following in dev mode after few minutes of work with a fair number of code changes and subsequent recompiling from the dev server:

WAIT Compiling... 23:17:38

[================== ] 90% (chunk assets processing)
<--- Last few GCs --->

[2864:000001CF89080F90] 1318476 ms: Mark-sweep 1397.3 (1448.1) -> 1397.3 (1448.1) MB, 364.2 / 0.0 ms allocation failure GC in old space requested
[2864:000001CF89080F90] 1318836 ms: Mark-sweep 1397.3 (1448.1) -> 1395.9 (1429.7) MB, 359.7 / 0.0 ms last resort GC in old space requested
[2864:000001CF89080F90] 1319125 ms: Mark-sweep 1395.9 (1429.7) -> 1395.9 (1429.2) MB, 288.5 / 0.0 ms last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 00000191239A5EE1
2: fromString(aka fromString) [buffer.js:314] [bytecode=0000039F4D2794E1 offset=164](this=00000060ACE82311 ,string=000000DEA4B96C99 <Very long string[7152076]>,encoding=00000191239B5CE1 <String[4]: utf8>)
4: from [buffer.js:177] [bytecode=0000039F4D278FE1 offset=11](this=0000024BC4D361B1 <JSFunction Buffer (sfi = 0000026D21602849)>,value=000000DEA4B96C99 <Very long string[71...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node_module_register
2: v8::internal::FatalProcessOutOfMemory
3: v8::internal::FatalProcessOutOfMemory
4: v8::internal::Factory::NewRawTwoByteString
5: v8::internal::Smi::SmiPrint
6: v8::internal::AllocationSpaceName
7: v8::String::WriteUtf8
8: v8::internal::PagedSpace::SetUp
9: node::Buffer::New
10: node::Buffer::New
11: v8::internal::CompilerDispatcherTracer::Estimate
12: v8::internal::CompilerDispatcher::RemoveJob
13: v8::internal::CompilerDispatcher::RemoveJob
14: v8::internal::CompilerDispatcher::RemoveJob
15: 000001DC663847A1

What were you expecting?

No errors.

What steps did you take, to get the error?

Nothing, just started a new quasar project as usual. quasar init etc...

Could be related to:
webpack/webpack#6389

I have 16 GB of RAM so memory was never an issue, maybe webpack 4 is hitting some treshold or just leaking memory.

[nothingismagick]

Thanks for reporting this. It is quite strange. Please upgrade NPM and NODE and let us know exactly which version of quasar you are running.

[danikane]

Quasar CLI is v0.16 - the latest. Both global and local. I was surprised that I was getting such errors so I looked up what's latest for Quasar on GitHub only to find it was released 4 hours ago. 😄

I can upgrade to Node v8.11.2 since it's the last LTS.

[danikane]

Ok, so I upgraded to Node v8.11.2 and NPM v6.0.1 and still managed to get that error by poking hard (switching back and forth between commits to trigger recompile). 😆
At some point Webpack really started to struggle (in terms of speed) to rebuild the app...

Since quasar-cli has grown into something bigger I can't really fiddle with webpack settings/plugins to test out different configurations.

Putting this one here just for reference:
webpack/webpack#6812

btw I'm using Yarn.

[nothingismagick]

@danikane Are you importing massively large files, doing stuff with breakpoints / styl ?

[danikane]

Nope, just a regular barebone Quasar project with:

    framework: 'all',
    animations: 'all',

I just edited the index.vue page. The very same configuration I have always used for a new project.
Same machine, same stuff, never had issues, always a great experience working with the Quasar framework.

I guess this issue hasn't gained much attention yet because people are still on 0.15.

[rstoenescu]

This is a webpack or a webpack-loader/plugin issue. Quasar CLI is not at fault here. Note that Quasar CLI v0.16 jumped to using webpack 4.

[danikane]

@rstoenescu That's exactly what I'm saying. :) I also think webpack 4 is the culprit here. Still Quasar CLI jumped on that wagon so I thought I'd let you know.

I watched the node dev server process spiking up in CPU usage and the memory footprint increased.
I guess webpack 3 had a better garbage collection mechanisms since several issues like this have been reported over time.
Could be uglify or postcss, who knows...

[nothingismagick]

webpack/webpack#7314

They suggest looking into adding the clean-webpack-plugin

[danikane]

I'd say this one:
webpack/webpack#6929
Some of the plugins are misbehaving...

[rstoenescu]

Would you be so kind as to make one small test, pls?
In your project folder, edit node_modules/quasar-cli/lib/build/webpack-config.js at line 297 (cache: true make it cache: false). This is just to determine if UglifyJS could be the culprit.

DO NOT run any npm/yarn commands after this change.

[danikane]

Razvan, I still managed to crash it. 😢 I also commented it out completely, but crashed again.
Right around it hits 1.6 GB memory usage it starts to slow down then all it takes are a few line changes and it crashes.
This time the stacktrace mentioned the html-webpack-plugin https://github.com/jantimon/html-webpack-plugin so I set its cache to false as well, but still the same result.

Remember, this is only on dev server, build passes ok, so right now this is minor inconvenience IMO.

[nothingismagick]

@danikane - Can you list all of the webpack plugins and babel loaders that you are using?

[danikane]

Do a **quasar init ** with Quasar CLI v0.16 and you'll have them.
Like I already said - fresh Quasar project with edits in index.vue, that's all.
I'm not using anything, because since v0.15 Webpack configuration is out of sight, embedded in the CLI.

[nothingismagick]

framework: all, animations:all -> that might be the cause of the issue here. And you potentially might need to increase your memory available to node...

[danikane]

All due respect, but I've always used framework: all, animations:all for development and never hit a snag.

You are right, node's memory limit is by default ~1.76 GB on x64, I would increase it if I could, except it has to be passed as a flag to the node executable and I don't know where to do it. 😄
If there is an option to do it via some environment variable ...

Theoretically even if I set it at 8 GB it would still hit the threshold at some point. BTW the projects starts at ~300 MB footprint.

[nothingismagick]

I am not trying to be disrespectful. :) But perhaps you remember the docs?
https://quasar-framework.org/components/introduction-for-beginners.html#Importing-All-Components-and-Directives-for-Quick-Test

framework: 'all'

IMPORTANT
This will not take advantage of tree shaking, causing your bundle to become bloated with unnescesary/unused code. Not recommended for production. Use this only for quick testing purposes.

[danikane]

Come on, man, I think I'm a reasonable and logical person. 😄 And I like to test thoroughly.
First of all thank you guys for trying to analyze the issue.

I just tested a fresh clean vue project (I know, it's small, doesn't have a huge amount of dependancies) and observed the memory usage. Same as Quasar CLI v0.15 it uses Webpack 3 (3.6.0 to be specific). Memory usage never went above 250 megs just because ... it cleans up well after itself and actually gets as low as 130, just because ... memory management.
I did the same amount of torture to a much bigger project using the v0.15 Quasar CLI.
Memory usage goes up of course, but then it goes down .. a lot!
With v0.16 it goes up big time then it goes down just a little bit, but overall it progressively increases. 😢
I can comment on memory usage all day long, but until someone else with v0.16 troubles chimes in it makes no sense. 😆

I'm not trying to bash you, Razvan or the repo, just pointing out an issue, trying to contribute some way or the other. I still think some plugin doesn't play well with Webpack 4 and I still think you guys did and are still doing great job, so thanks again!

[nothingismagick]

This is the ENV var you can use as a temporary workaround until we resolve the issue properly:

https://nodejs.org/dist/latest-v8.x/docs/api/cli.html#cli_node_options_options
NODE_OPTIONS=--max_old_space_size=4096

There are however issues with this:
cypress-io/cypress#1676
electron/electron#12695

[nothingismagick]

I just tested that like this:

$ NODE_OPTIONS=--max_old_space_size=4096 DEBUG=v8:* npm run dev:pwa

where dev:pwa is "dev:pwa": "quasar dev -m pwa", in package.json

[nothingismagick]

Over at the discord server several people have also reported this.
My hunch is that something is wrong in the source-map process

[nothingismagick]

$ npm ls source-map
[email protected] /project
├─┬ [email protected]
│ └─┬ @babel/[email protected]
│   └─┬ @babel/[email protected]
│     └── [email protected]  deduped
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │   └── [email protected]  deduped
│ └─┬ [email protected]
│   └─┬ [email protected]
│     └─┬ [email protected]
│       └── [email protected]  deduped
├─┬ [email protected]
│ └─┬ [email protected]
│   ├─┬ [email protected]
│   │ ├─┬ [email protected]
│   │ │ └── [email protected]  deduped
│   │ └─┬ [email protected]
│   │   └─┬ [email protected]
│   │     ├── [email protected] 
│   │     └─┬ [email protected]
│   │       └── [email protected] 
│   ├─┬ [email protected]
│   │ └── [email protected]  deduped
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   ├─┬ [email protected]
│   │   │ ├─┬ [email protected]
│   │   │ │ └── [email protected]  deduped
│   │   │ └─┬ [email protected]
│   │   │   └── [email protected]  deduped
│   │   └── [email protected]  deduped
│   └─┬ [email protected]
│     └── [email protected] 
├─┬ [email protected]
│ └─┬ [email protected]
│   └── [email protected] 
├─┬ [email protected]
│ └─┬ [email protected]
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └─┬ [email protected]
│   │     └─┬ [email protected]
│   │       └── [email protected]  deduped
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └── [email protected] 
│   ├─┬ [email protected]
│   │ └── [email protected]  deduped
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └── [email protected] 
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └── [email protected] 
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └── [email protected] 
│   └─┬ [email protected]
│     └─┬ [email protected]
│       └── [email protected] 
├─┬ [email protected]
│ └─┬ [email protected]
│   └── [email protected] 
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │   └── [email protected]  deduped
│ ├─┬ [email protected]
│ │ └── [email protected] 
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │   ├── [email protected] 
│ │   └─┬ [email protected]
│ │     └── [email protected]  deduped
│ ├─┬ [email protected]
│ │ └── [email protected] 
│ └─┬ [email protected]
│   └─┬ [email protected]
│     └── [email protected]  deduped
└─┬ [email protected]
  ├─┬ @babel/[email protected]
  │ ├─┬ @babel/[email protected]
  │ │ └── [email protected]  deduped
  │ └── [email protected] 
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   └── [email protected] 
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   ├─┬ [email protected]
  │   │ └── [email protected]  deduped
  │   └─┬ [email protected]
  │     └── [email protected] 
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   └── [email protected] 
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   └─┬ [email protected]
  │     └── [email protected] 
  ├─┬ [email protected]
  │ └── [email protected] 
  ├─┬ [email protected]
  │ └── [email protected] 
  ├─┬ [email protected]
  │ └── [email protected] 
  └─┬ [email protected]
    └─┬ @vue/[email protected]
      ├─┬ [email protected]
      │ └── [email protected] 
      ├─┬ [email protected]
      │ └── [email protected] 
      └── [email protected]  deduped

[danikane]

Thanks, I set the env.variable.
Yes, source maps were mentioned in the webpack threads. 👍

[nothingismagick]

I wonder if this is our problem:

  ├─┬ [email protected]
  │ └── [email protected] 

[nothingismagick]

Stylus is a bit stale...

[danikane]

Yep, right:

{
"name": "stylus",
"description": "Robust, expressive, and feature-rich CSS superset",
"version": "0.54.5",
...
},
"dependencies": {
"css-parse": "1.7.x",
"mkdirp": "0.5.x",
"debug": "*",
"sax": "0.5.x",
"glob": "7.0.x",
"source-map": "0.1.x"
},
...
}

[nothingismagick]

https://github.com/stylus/stylus/blob/dev/package.json

  "dependencies": {
    "css-parse": "~2.0.0",
    "debug": "~3.1.0",
    "glob": "~7.1.2",
    "mkdirp": "~0.5.x",
    "sax": "~1.2.4",
    "source-map": "~0.7.2"
  },

[danikane]

Yes, a lot of commits since the last release, so we wait for a new release and update Quasar CLI's dependancies?

[nothingismagick]

Worst practice would be to remove the bad version of source-map in that library's node modules, and copy + paste a version from e.g. uglify-es/node_modules which I just did. And quasar devstill runs for me...

[nothingismagick]

(and first compile is also a bit faster, I must say...)

[danikane]

Yeah, but GLOBAL mode is not yet implemented. 😃

[nothingismagick]

I'll leave GLOBAL for future work. And by the way, I actually used this on another project today and it worked perfectly.

[rstoenescu]

@danikane One more test please. Without any other temporary solutions in this thread, can you set quasar init then set quasar.conf > build > sourceMap: false and see if it still happens? Thank you for your efforts!

[danikane]

Hey, glad to help!
Umm, it still does although not so dramatically. Does occasionally free up a little bit of RAM inbetween changes... So in a way - it's effective.
I quasar init, did what you suggested, didn't touch anything except the default layout, made a lot of consecutive changes, still eats it up. 😢

Another project eventually drops to the initial memory consumption.

[danikane]

I was comparing the Sources tab on Chrome on two different projects and it seemed strange that the Vue file (from sourcemap) I'm editing doesn't get updated at all when updating the template (on the latest quasar compared to fresh vue project for example), only webpack hot update scripts get pushed.
When updating the script part though it refreshes and shows the edit correctly.

These are some of the dependancies of a pure vue project:

"vue-loader": "^13.3.0",
"vue-style-loader": "^3.0.1",
"vue-template-compiler": "^2.5.2",

And these are quasar's:

"vue-loader": "15.1.0",
"vue-style-loader": "4.1.0",
"vue-template-compiler": "2.5.16",

But that's really black magic for me. 😄

Or maybe Vue team just didn't update their template.
https://github.com/vuejs-templates/webpack

[panstromek]

Just wanted to mention that dev server on my co-workers Mac sometimes randomly crashes for the same reason (Allocation failed, heap out of memory).

[nothingismagick]

@panstromek - have you tried this:

$ NODE_OPTIONS=--max_old_space_size=4096 DEBUG=v8:* npm run dev:pwa

where dev:pwa is "dev:pwa": "quasar dev -m pwa", in package.json

[nothingismagick]

So this is still happening, btw. And now the following detail came to light on Discord:

"I get that sometimes.. usually thanks to hot-reload in the middle of broken code being saved. Just restart quasar dev."

I replied: " the interrupted hot-reload would explain this. The cache is not invalidated (GC'd) and won't be until webpack completes its run - so it just explodes and the node parent process can't allocate any more buffers because they're all full and locked."

[rstoenescu]

So it means it must be fixed within Webpack package and Quasar can't do anything about it. Opening a ticket on webpack or webpack-dev-server repository (with prior check if it's already there something similar) should be the way to go further.

[nothingismagick]

@rstoenescu - I opened an issue here about the source-map problem: stylus/stylus#2386

nothingismagick commented 7 days ago •
webpack/webpack#7314
They suggest looking into adding the clean-webpack-plugin

@danikane
danikane commented 7 days ago •
I'd say this one:
webpack/webpack#6929
Some of the plugins are misbehaving...

[rstoenescu]

clean-webpack-plugin does not interfere with webpack dev mode, so not a solution. The problem in webpack is somewhere else.

[rstoenescu]

@danikane can you update your node version to >= 10 and retry this? It seems like there would be a problem with node 8 only. Sorry for repeatedly asking to test.

[leo-buneev]

Hi! Did some testing of various configs related to this issue on pretty small project:

❌ Using node 10.0 did not fix the issue
❌ Disabling source maps did not fix the issue, although significantly increased time that it takes to fail
❌ Using superdep surgical on stylus did not fix the issue
✔️ Finally, disabling PostCSS (by removing all PostCSS loaders from generated webpack config) fixed the issue.

Honestly, I would appreciate possibility to disable PostCss in quasar.conf.js regardless - ran into some non-trivial problems introduced by PostCss on quasar 0.14 and needed to disable it.

Workaround for people that want to try disabling PostCss now:

// quasar.conf.js
extendWebpack(cfg) {
  for (const rule of cfg.module.rules) {
    if (!rule.oneOf) continue
    for (const ruleOneOf of rule.oneOf) {
      ruleOneOf.use = ruleOneOf.use.filter(u => u.loader !== 'postcss-loader')
    }
  }
}

[monoc44]

Ran into same problems. I confirm: use of node 10 didn’t fix the issue.

Thanks @leo-buneev, will try your workaround.

[monoc44]

Sadly, disabling PostCss didn't fix the problem for me:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

[Eldar-X]

I have the latest versions of all packages and node.js but occasionally I have the same problems.

[nothingismagick]

@Eldar-X - can you be more specific about the situation surrounding your experience with this issue? Does it happen when a hot-reload is interrupted with new changes?

[rstoenescu]

Made a small change to Quasar CLI (will be available in 0.16.5), based on one comment from sokra: webpack/webpack#1914 (comment)

I'm still not confident this fixes it.
Seems like most Webpack projects are affected by this... vue-cli included.

[nothingismagick]

The link to the similar issue at vue is here: vuejs/vue-cli#1453

[boid-com]

I'm suffering from this issue as well. I have noticed that the dev server will crash more often when editing css. Also it tends to recompile more slowly over time before finally crash. previous versions of quasar didn't have this issue. seems like a huge oversight from Webpack team?

[juanigonvocal]

Hi, any updates on this? Tried all the workarounds suggested, but still getting this error when building on my cloud server:

 [==================  ] 90% (chunk assets processing)node[576]: pthread_create: Resource temporarily unavailable

[nothingismagick]

@juanigonvocal - that is something else. This issue is about the dev server running out of memory. Your problem appears to be that the user cannot assign a thread to the node process.

[juanigonvocal]

@nothingismagick yes, sorry I created #162 Thanks!

[danikane]

Hey guys, let me report back after being away for quite some time. 😃
Using versions below it's still looking fresh after few hours of editing. Memory usage is normal, no errors whatsoever.

N App URL........... http://localhost:8080/
Dev mode.......... spa
Quasar theme...... mat
Quasar CLI........ v0.17.11
Quasar Framework.. v0.17.9

Soooo, I guess this could be considered case closed or at least for the v0.17.x CLI.
Thanks everyone!

[nothingismagick]

Then by all means - close it!

[skirankumar7]

$ npm ls source-map
[email protected] /project
├─┬ [email protected]
│ └─┬ @babel/[email protected]
│   └─┬ @babel/[email protected]
│     └── [email protected]  deduped
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │   └── [email protected]  deduped
│ └─┬ [email protected]
│   └─┬ [email protected]
│     └─┬ [email protected]
│       └── [email protected]  deduped
├─┬ [email protected]
│ └─┬ [email protected]
│   ├─┬ [email protected]
│   │ ├─┬ [email protected]
│   │ │ └── [email protected]  deduped
│   │ └─┬ [email protected]
│   │   └─┬ [email protected]
│   │     ├── [email protected] 
│   │     └─┬ [email protected]
│   │       └── [email protected] 
│   ├─┬ [email protected]
│   │ └── [email protected]  deduped
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   ├─┬ [email protected]
│   │   │ ├─┬ [email protected]
│   │   │ │ └── [email protected]  deduped
│   │   │ └─┬ [email protected]
│   │   │   └── [email protected]  deduped
│   │   └── [email protected]  deduped
│   └─┬ [email protected]
│     └── [email protected] 
├─┬ [email protected]
│ └─┬ [email protected]
│   └── [email protected] 
├─┬ [email protected]
│ └─┬ [email protected]
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └─┬ [email protected]
│   │     └─┬ [email protected]
│   │       └── [email protected]  deduped
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └── [email protected] 
│   ├─┬ [email protected]
│   │ └── [email protected]  deduped
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └── [email protected] 
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └── [email protected] 
│   ├─┬ [email protected]
│   │ └─┬ [email protected]
│   │   └── [email protected] 
│   └─┬ [email protected]
│     └─┬ [email protected]
│       └── [email protected] 
├─┬ [email protected]
│ └─┬ [email protected]
│   └── [email protected] 
├─┬ [email protected]
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │   └── [email protected]  deduped
│ ├─┬ [email protected]
│ │ └── [email protected] 
│ ├─┬ [email protected]
│ │ └─┬ [email protected]
│ │   ├── [email protected] 
│ │   └─┬ [email protected]
│ │     └── [email protected]  deduped
│ ├─┬ [email protected]
│ │ └── [email protected] 
│ └─┬ [email protected]
│   └─┬ [email protected]
│     └── [email protected]  deduped
└─┬ [email protected]
  ├─┬ @babel/[email protected]
  │ ├─┬ @babel/[email protected]
  │ │ └── [email protected]  deduped
  │ └── [email protected] 
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   └── [email protected] 
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   ├─┬ [email protected]
  │   │ └── [email protected]  deduped
  │   └─┬ [email protected]
  │     └── [email protected] 
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   └── [email protected] 
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   └─┬ [email protected]
  │     └── [email protected] 
  ├─┬ [email protected]
  │ └── [email protected] 
  ├─┬ [email protected]
  │ └── [email protected] 
  ├─┬ [email protected]
  │ └── [email protected] 
  └─┬ [email protected]
    └─┬ @vue/[email protected]
      ├─┬ [email protected]
      │ └── [email protected] 
      ├─┬ [email protected]
      │ └── [email protected] 
      └── [email protected]  deduped

what does this means could you explain once......

[panstromek]

@skirankumar7 It's a tree of all installed packages (organized by dependencies) that contain source-map as dependency