File is ignored after removing and quick adding back

[vojtajina]

Ubuntu 12.04.1 LTS
Chokidar 0.5.1
Node 0.8.14

var w = chokidar.watch('watched');
['add', 'change', 'unlink'].forEach(function(event) {
  w.on(event, function(p) {
    console.log('chokidar:', event, p);
  });
});

In command line (this basically simulates VIM's backup behavior):

cd watched
mv some.js some.js.swp && sleep .1 && mv some.js.swp some.js

Output:

chokidar: change watched/some.js
chokidar: unlink watched/some.js
chokidar: add watched/some.js.swp
chokidar: change watched/some.js.swp

Chokidar does not catch when some.js is moved back. And any subsequent changes on some.js are ignored as well.

If I increase the sleep between mv commands, it works fine. Node's fs.watchFile does not watch a file, once it's removed (eg. adding the file back after removing is ignored). Chokidar catches new files by watching the directory. So I guess there is some sort of race condition between the "directory watcher" and "file" watcher - the "directory" watcher needs to be fired after the "file remove" watcher, so that it registers a new watcher for the file - I guess this doesn't happen.

Based on https://github.com/vojtajina/testacular/issues/199

[vojtajina]

Ok, I looked into it a bit...

The "directory watcher" (when the file is added back) is ignored because of this condition https://github.com/paulmillr/chokidar/blob/master/src/index.coffee#L120 (the timestamps as the same).

So it does not fire the callback, that would compare the content. If I remove this condition, it works fine.

We don't wanna remove this condition, that would mean a lot of unnecessary operations. Is there anything else we could quickly compare, instead of mtime ? Or maybe we could keep this condition for files and always fire the callback for directories and try to optimize the comparison a bit (https://github.com/paulmillr/chokidar/blob/master/src/index.coffee#L144-L162).

[paulmillr]

moreover, removing of the condition will lead to many bugs. did this in the past. mtime is the most useful stuff.

optimising comparing of file trees seems to be cooler solution.

[vojtajina]

Well, optimizing the comparison won't fix it - that's in addition to removing the condition for directories.

What bugs would this lead into ?

[paulmillr]

I don’t remember exactly, but there was some cross-platform issues, events were not emitted on windows / os x etc or emitted pretty shitty, not every time

[amj]

For people noticing this effect from using vim, the following addition to your .vimrc is a temporary workaround:
:set backupcopy=yes

(from the discussion on the previously closed issue here: karma-runner/karma#199)

[paulmillr]

Just updating current status: you can use fs.watch currently with option, but it is much more unreliable.

[es128]

Just throwing an idea out there, even though I haven't studied this too closely:

What if the remove operations were put behind a quick setTimeout, and then conditioned by a fs.exists check? Will anyone really be bothered by a (let's say) 50ms delay on the unlink event being fired?

[paulmillr]

not sure it will solve the problem

[cesutherland]

I believe this is the correct workaround for vim:

set nowritebackup

With this option, vim will write the buffer to the file directly instead of 1. write to backup, 2. delete original, 3. rename backup. This means risking data loss in the event of an IO error.

[es128]

95c6dfc solves this for fsevents, 8f41a70 solves it for polling. The fs.watch method seemed to work for this situation already.

Duplicate change events may still happen, but with vim I've observed them happening anywhere between 30ms and 500ms apart from each other, so I don't think it's reasonably possible or desirable to suppress them, since it's an accurate reflection of what's happening with the file.

[cesutherland]

🤘

[jugheadjones10]

Hi @es128 , I was using Tailwindcss, which uses Chokidar, and I came across this same problem. Is there a possibility that this issue isn’t resolved yet? Thanks.

[paulmillr]

@jugheadjones10 there is no point in asking this question a person from 2015.

[jugheadjones10]

@jugheadjones10 there is no point in asking this question a person from 2015.

@paulmillr Then can I direct it to you instead? Thanks! If needed I might be interested in creating a PR for this.

[cesutherland]

2015 was pretty dope tho

[paulmillr]

@cesutherland in some way, yes