EMFILE or ENOENT when building for osx

[jthrilly]

For my app building for win32 and win64 works fine, but building for any osx platform will result in either an EMFILE or ENOENT error.

I'm running Yosemite 10.10.2, so having the usual trouble with ulimit not working. However, launchctl limit maxfiles reports:
maxfiles 10240 10240
and ulimit -n reports 1024.

Examples of errors:

joshua@why ~> node /usr/local/bin/nwbuild -p osx64 ~/Sites/networkCanvas/ -o ~/Sites/networkCanvas/build/
Latest Version: v0.12.0-alpha2
Using v0.12.0-alpha2
Create cache folder in /usr/local/lib/node_modules/node-webkit-builder/cache/0.12.0-alpha2
Using cache for: osx64
Create release folder in /Users/joshua/Sites/networkCanvas/build/networkCanvas/osx64
[ { [Error: EMFILE, open '/Users/joshua/Sites/networkCanvas/build/networkCanvas/osx64/networkCanvas.app/Contents/Resources/app.nw/bower_components/jquery-ui/ui/effect-size.js']
    errno: 20,
    code: 'EMFILE',
    path: '/Users/joshua/Sites/networkCanvas/build/networkCanvas/osx64/networkCanvas.app/Contents/Resources/app.nw/bower_components/jquery-ui/ui/effect-size.js' } ]

joshua@why ~> node /usr/local/bin/nwbuild -p osx64 ~/Sites/networkCanvas/ -o ~/Sites/networkCanvas/build/
Latest Version: v0.12.0-alpha2
Using v0.12.0-alpha2
Create cache folder in /usr/local/lib/node_modules/node-webkit-builder/cache/0.12.0-alpha2
Using cache for: osx64
Create release folder in /Users/joshua/Sites/networkCanvas/build/networkCanvas/osx64
{ [Error: ENOENT, no such file or directory '/Users/joshua/Sites/networkCanvas/build/networkCanvas/osx64/networkCanvas.app/Contents/Resources/app.nw/README.html']
  errno: 34,
  code: 'ENOENT',
  path: '/Users/joshua/Sites/networkCanvas/build/networkCanvas/osx64/networkCanvas.app/Contents/Resources/app.nw/README.html',
  syscall: 'lstat' }

[adam-lynch]

Which version of node-webkit-builder have you?

[jthrilly]

I'm afraid I can't access the version number because of the line endings issue reported elsewhere. No matter which way I call the file from terminal, I get env: node\r: No such file or directory

Sorry I can't be more useful!

[adam-lynch]

@jthrilly you can see in node_modules/node-webkit-builder/package.json

[danielhusar]

I have exactly same error, my version of node-webkit-builder is 1.0.8

[brian-mann]

I am also receiving the identical error message with version 1.0.8

[mario-deluna]

Same here for 1.0.8.

[Deliaz]

Have the same (1.0.8)
But it happens not on Mac Os only. If I build app for Mac on Windows 8.1, output is:

So, I get successful build, if exclude 'jquery' module from files to build.
But app doesn't work properly, cuz jquery requires..

[elliotbonneville]

Experiencing the same issue as well.

[adam-lynch]

Could someone try again with 1.0.10?

[gihrig]

1.0.10 fixed this issue for me, I guess. My app now builds for OS X (OS X 10.10.2) without error, but it does not execute.

On execution, console log shows:

com.apple.xpc.launchd[1]: (com.node-webkit-builder.icrumz.291132[939]) Service exited with abnormal code: 1

May be unrelated, I'm still researching

[philipproplesch]

I can confirm that 1.0.10 solved this issue for me on Windows 👍

[danielhusar]

For me it works fine now on osx. Thanks.

[adam-lynch]

Closing. I'll reopen if this happens for anyone still.

[benjismith]

I just updated to 1.0.11 this morning, and I'm still seeing EMFILE errors during my build. I'm on Mac OS X Yosemite 10.10.2.

[adam-lynch]

[benjismith]

There there... It will all be better someday :)

Let me know if there's anything I can do to help. Do you need any logfiles or anything?

[kessler]

@benjismith Is the error consistent (i.e happens every time)?

[benjismith]

Yes, it happens every time.

[kessler]

Is is possible for you to paste your package.json here, specifically the dependencies?

[benjismith]

{
  "name": "MyApplication",
  "version": "0.0.1",
  "private": true,
  "main": "app/index.html",
  "single-instance" : true,
  "window": {
    "width": 1280,
    "height": 750,
    "toolbar": false
  },
  "dependencies": {
    "MD5": "latest",
    "moment": "latest",
    "hogan.js": "latest",
    "lodash-node": "latest",
    "mkdirp": "latest",
    "form-data" : "latest",
    "node-webkit-updater" : "latest"
  },
  "devDependencies": {
    "gulp": "^3.8.11",
    "gulp-util": "^3.0.4",
    "node-webkit-builder": "^1.0.11"
  }
}

[kessler]

So the builder sits in the same directory as the application? (you have one or two package.json)

[benjismith]

It looks like this:

MyApplication/package.json
MyApplication/GulpFile.js
MyApplication/app/index.html
MyApplication/app/main.js
MyApplication/app/vendor/     # jquery, bootstrap, etc (~1500 files)
MyApplication/app/styles/     # app-specific css (~20 files)
MyApplication/app/lib/        # app-specific javascript (~100 files)

[kessler]

Was hoping you'll have two. In any case, can you try deleting all the non dev dependencies and run the build?

[benjismith]

I tried deleting the dependencies and building again, and it doesn't help. I also tried building on windows, and that doesn't help either.

It seems like the root cause of the problem here is trying to simultaneously copy thousands of files using async operations. Is that really necessary? Is there any advantage? Why not just loop through the files and copy them synchronously, one at a time?

[agozie]

Same EMFILE error on version 1.1.0, on Yosemite and Ubuntu 14.10.

[chino23]

Same here OSX 10.10.2 and node-webkit-builder 1.0.11.

Had the EMFILE error 20, then after updating node to 0.12.0 and npm to 2.7.0 I have the same error, but with error number -24

My app folder has 2063 files. Interesting, cause I have some npm scripts and gulp scripts that copy the whole folder and this never happens.

[adam-lynch]

@chino32 try the latest one

On Thu, 12 Mar 2015 09:39 chino23 [email protected] wrote:

Same here OSX 10.10.2 and node-webkit-builder 1.0.11.

Had the EMFILE error 20, then after updating node to 0.12.0 and npm to
2.7.0 I have the same error, but with error number -24

My app folder has 2063 files. Interesting, cause I have some npm scripts
and gulp scripts that copy the whole folder and this never happens.

—
Reply to this email directly or view it on GitHub
#180 (comment)
.

[chino23]

hmm @adam-lynch am I missing something? :) I'm already using the latest version of node-webkit-builder, and there hasn't been any changes in the last 24 hours as I can see.

[adam-lynch]

My mistake, @chino23. I was thinking the latest was 1.1.0.

[fernandogrd]

Same here, Linux 3.18.6, node-webkit-builder 1.0.11.
I had to raise ulimit to 8192 to be able to run, but I would like to run on a CI environment where I don't have permissions to change ulimit (there the ulimit is 4096)

[bastimeyer]

Could this be related to #204 ?

[eugirdor]

I started getting ENFILE errors after adding a particularly large library to my project. I have traced down the issue and it appears to be caused by this code in NwBuilder.mergeAppFiles():

https://github.com/mllrsohn/node-webkit-builder/blob/ecc96607491212c6568c0753847dc622255889d4/lib/index.js#L433-L442

If I understand the code correctly, that line attempts to copy every file in the project simultaneously. You might get lucky enough for some of the files to finish before it starts more, but eventually, you will hit the limit of how many open files you can have. I made a fix locally, limiting the number of concurrent copies and no longer have the issue.

@adam-lynch If you don't mind bluebird being updated to 2.x.x, I can submit a PR (I use the 2.0 concurrency option in Promise.map for my fix.)

[chino23]

I would love a fix for that!

[mmfilesi]

Similar error, sorry. Windows 64.

Running "nwjs:src" (nwjs) task
Latest Version: v0.13.0-beta2
Using v0.13.0-beta2
Create cache folder in c:\Users\marcos\documents\jsLabs\library\cache\0.13.0-beta2
Downloading: http://dl.nwjs.io/v0.13.0-beta2/nwjs-v0.13.0-beta2-win-ia32.zip
Create cache folder in c:\Users\marcos\documents\jsLabs\library\cache\0.13.0-beta2
Downloading: http://dl.nwjs.io/v0.13.0-beta2/nwjs-v0.13.0-beta2-win-x64.zip
                                                downloading [====---------------  downloading [====================] 100% 0.0s

Create release folder in c:\Users\marcos\documents\jsLabs\library\webkitbuilds\library\win32
Create release folder in c:\Users\marcos\documents\jsLabs\library\webkitbuilds\library\win64
Fatal error: ENOENT: no such file or directory, lstat 'c:\Users\marcos\documents\jsLabs\library\cache\0.13.0-beta2\win32\ffmpegsumo.dll'

package.json

  "devDependencies": {
    "grunt": "^0.4.5",
    "grunt-nw-builder": "^2.0.0",
    "load-grunt-tasks": "^3.4.0",
    "nw": "^0.12.3",
    "nw-builder": "^2.2.0"
  }

[adam-lynch]

@mmfilesi see #275. Specify a non-alpha version and should be fine.

[mmfilesi]

I @adam-lynch, ty.

I try now with "nw": "0.12.2",

but the result...

Fatal error: ENOENT: no such file or directory, lstat 'c:\Users\marcos\documents\jsLabs\libr\cache\0.13.0-beta2\win32\ffmpegsumo.dll'

[adam-lynch]

@mmfilesi oh no, what I mean is when you're calling nw-builder, you need to pass a version;

var NwBuilder = require('nw-builder');
var nw = new NwBuilder({
    files: './path/to/nwfiles/**/**', // use the glob format
    platforms: ['osx32', 'osx64', 'win32', 'win64'],
    version: '0.12.2'
});

That should work fine.

[mmfilesi]

Thats work fine : ) : ) : )

Ty adam-lynch, !!!!

[adam-lynch]

I haven't heard about this problem in a long long time. I think it's ok. Fixes have also been released today by the way (newer NW.js versions are support, alpha/beta builds, etc.)