Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs.copy silently fails on windows #358

Closed
aeneasr opened this issue Feb 16, 2017 · 24 comments
Closed

fs.copy silently fails on windows #358

aeneasr opened this issue Feb 16, 2017 · 24 comments

Comments

@aeneasr
Copy link

aeneasr commented Feb 16, 2017

This is an upstream issue report from electron-packager#544 and in particular this comment. The issue exists in both 1.0 and 2.0 versions.

electron-packager is a cli helper that packages an electron app. It does so by copying a node app (including node_modules) to the temp directory. This step uses fs.copy which silently fails. Neither the callback is called, nor an error is thrown.

This is the line which fails silently

fs.copy(opts.dir, appPath, {filter: ignore.userIgnoreFilter(opts), dereference: shouldDeref}, cb)

I'd be glad to help you identify root cause of this issue. My current guess that it's combination of filter/dereference and maybe (?) long paths?

Note that this only happens on windows. On OSX, packaging the project works fine.

@jprichardson
Copy link
Owner

@arekkas thank you for your report and offer to help.

It seems that fs.copy() all of a sudden is failing for a large swath of people. See in particular:

Let's start with this... which version of Node are you using? Does changing the version of Node to a previous version (in particular, older major), still cause the issue?

@aeneasr
Copy link
Author

aeneasr commented Feb 16, 2017

Thanks for your swift reply. I'm currently using node 6.6.0. Unfortunately, there is no nvm I am aware of that works on windows. So up-/downgrading is a bit of a pain. I'd do it however if you're confident that that might be the issue.

@jdalton
Copy link
Contributor

jdalton commented Feb 16, 2017

Unfortunately, there is no nvm I am aware of that works on windows

btw I use nvm with BashOnWindows in Windows 10.

@jprichardson
Copy link
Owner

@arekkas https://github.com/coreybutler/nvm-windows However, please only switch nvm windows if you think you'll get personal value out of it. Thanks for your willingness to help here.

@aeneasr
Copy link
Author

aeneasr commented Feb 16, 2017

Ok, installed. Tried majors 4, 5, 6. All of them have (apparently) the same issue.

@aeneasr
Copy link
Author

aeneasr commented Feb 16, 2017

I did some good old console.log debugging. @jprichardson it looks like the culprit is the counter in ncp.js:

  function doneOne (skipped) {
    console.log('doneOne 1', skipped)
    if (!skipped) running--
    console.log('doneOne 2', skipped)
    finished++
    if ((started === finished) && (running === 0)) {
      console.log('doneOne 3', skipped)
      if (callback !== undefined) {
        console.log('doneOne 4', skipped)
        return callback(null)
      }
    }
    console.log('doneOne 5', skipped, started, finished, running)
  }

The last logs before the task quits are:

...
doneOne 5 undefined 40369 24699 15670
doneOne 1 undefined
doneOne 2 undefined
doneOne 5 undefined 40369 24700 15669
doneOne 1 undefined
doneOne 2 undefined
doneOne 5 undefined 40369 24701 15668
doneOne 1 undefined
doneOne 2 undefined
doneOne 5 undefined 40369 24702 15667
doneOne 1 undefined
doneOne 2 undefined
doneOne 5 undefined 40369 24703 15666
doneOne 1 undefined
doneOne 2 undefined
doneOne 5 undefined 40369 24704 15665
doneOne 1 undefined
doneOne 2 undefined
doneOne 5 undefined 40369 24705 15664

@aeneasr
Copy link
Author

aeneasr commented Feb 16, 2017

Ok, further debugged it. Apparently the last operation that is happening is copyFile. It always stops at the same file:

copyFile { name: 'C:\\workspace\\js\\sites-app\\node_modules\\browser-resolve\\node_modules\\resolve\\test\\subdirs\\node_modules\\a\\b\\c\\x.json',
  mode: 33206,
  mtime: 2017-02-16T20:11:38.580Z,
  atime: 2017-02-16T20:11:38.574Z,
  stats:
   { dev: -325773423,
     mode: 33206,
     nlink: 1,
     uid: 0,
     gid: 0,
     rdev: 0,
     blksize: undefined,
     ino: 3940649675519715,
     size: 8,
     blocks: undefined,
     atime: 2017-02-16T20:11:38.574Z,
     mtime: 2017-02-16T20:11:38.580Z,
     ctime: 2017-02-16T20:11:38.580Z,
     birthtime: 2017-02-16T20:11:38.574Z } } D:\Temp\electron-packager\linux-ia32\ORY Sites App-linux-ia32\resources\app\node_modules\browser-resolve\node_modules\resolve\test\subdirs\node_modules\a\b\c\x.json

@aeneasr
Copy link
Author

aeneasr commented Feb 16, 2017

On a side note: it would help so much if there was debugging information available from node-fs-extra using an env var or something :D

@aeneasr
Copy link
Author

aeneasr commented Feb 17, 2017

@jprichardson is it possible that this is caused by the stat part? From looking over the code it seems like the only place where it could happen that the counter gets increased without a chance of being decreased if stat isn't calling the callback:

stat(source, function (err, stats) {

I fucking hate windows.

@RyanZim
Copy link
Collaborator

RyanZim commented Feb 17, 2017

@jprichardson Sometimes I wonder if we shouldn't just escalate #292.

With native promise support in node 4+, messing with these counters just seems so wrong.

@aeneasr
Copy link
Author

aeneasr commented Feb 19, 2017

So I'm not really sure where the culprit is now, if it's node-fs-extra or node-graceful-fs. Anyone any ideas? Unfortunately it's a blocking issue for me :(

@jprichardson
Copy link
Owner

So I'm not really sure where the culprit is now, if it's node-fs-extra or node-graceful-fs. Anyone any ideas? Unfortunately it's a blocking issue for me :(

Not yet, but we're digging! Hopefully can get a resolution early this week.

@aeneasr
Copy link
Author

aeneasr commented Feb 20, 2017

The issue does apparently no longer occur on node 7.6.0

@manidlou
Copy link
Collaborator

@arekkas are you still experiencing the issue?

@aeneasr
Copy link
Author

aeneasr commented Feb 22, 2017

Not with node 7.6.0 :)

@jprichardson
Copy link
Owner

@arekkas since Node v7.6.0 fixes things, perhaps it was a libuv issue? Are you comfortable with us closing this issue?

@aeneasr
Copy link
Author

aeneasr commented Feb 24, 2017

Maybe it could be pointed out in the readme! Otherwise good to close for me.

@RyanZim
Copy link
Collaborator

RyanZim commented Feb 24, 2017

SGTM, @arekkas care to submit a PR?

@ghost
Copy link

ghost commented Feb 26, 2017

is there a specific change in node 7.6.0 that fixed it? If so, then shouldn't the node folks be backporting it to the LTS release?

@RyanZim
Copy link
Collaborator

RyanZim commented Feb 27, 2017

@jrobeson We're not quite sure what the root of the issue is here. It may be an issue with Node, or it may be an issue with the way graceful-fs works on Node <v7.

@nabito
Copy link

nabito commented Jun 22, 2017

I'm experiencing fse silently failed too, but my settings is different.

Everything is running on Linux with no problem, until I switch my working terminal to be Windows (node is still running on Linux server), update a few files using local editor, and boom! fse.copy() doesn't work anymore.

Note that it's fse.copy() on Node v8.0.0 on Ubuntu.

Trial and errors resolved the problem, the suspect was line ending characters on Windows has creep into some files I edited locally. After removing them, fse.copy() work partially.

I'm using pm2-dev cmd to start my node app, and the problem only appear when invoking pm2-dev not normal production pm2.

Not sure yet what causes the problem but giving you guys my 2 hints.

@rikumi
Copy link

rikumi commented Feb 1, 2018

I got the same error on macOS now. After called by Electron-packager and copied part of the files, fs-extra silently quitted without calling the callback. Also, the last copied file varies and doesn't seem to be the cause.

@RyanZim
Copy link
Collaborator

RyanZim commented Feb 5, 2018

We're now on v5, and copy has changed a lot since then. Assuming this is fixed and closing.

@RyanZim RyanZim closed this as completed Feb 5, 2018
@girlzap
Copy link

girlzap commented Apr 24, 2018

Can confirm this issue seems to have been fixed with v5 of fs-extra.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants