fs.watch() unreliable when using vim

[scribu]

I'm trying to build a continuous recompilation utility for LESS files and I'm getting errant results.

Sample code:

fs.watch('example.less', function(event, fname) {
  console.log(+(new Date), arguments);
});

First file edit: nothing.
Second file edit:

1335280930608 { '0': 'change', '1': 'example.less' }
1335280930610 { '0': 'rename', '1': 'example.less' }
1335280930610 { '0': 'rename', '1': 'example.less' }

Subsequent edits: nothing.

My environment: vim 7.3, Ubuntu 11.10, node 0.6.15

[bnoordhuis]

vim probably does a save and rename. Do you see similar behavior with inotifywait in monitor mode?

[scribu]

inotifywait -m core.less:

First edit:

core.less MOVE_SELF

Second edit:

core.less ATTRIB 
core.less DELETE_SELF 

Later edits: nothing

[bnoordhuis]

Interesting. Can you trace your script with strace -f -o trace.log node script.js and gist trace.log? I would like to know if the initial event is picked up by libuv.

[scribu]

Here you go: https://gist.github.com/14fab4b84cf8a85bdec6

[bnoordhuis]

Thanks. Can you try this patch?

diff --git a/deps/uv/src/unix/linux.c b/deps/uv/src/unix/linux.c
index 809e644..a01d14b 100644
--- a/deps/uv/src/unix/linux.c
+++ b/deps/uv/src/unix/linux.c
@@ -271,6 +271,7 @@ int uv_fs_event_init(uv_loop_t* loop,
          | IN_MODIFY
          | IN_DELETE
          | IN_DELETE_SELF
+         | IN_MOVE_SELF
          | IN_MOVED_FROM
          | IN_MOVED_TO;

[scribu]

Ok, patched and rebuilt v0.6.15.

First edit:

1335309471878 { '0': 'rename', '1': 'example.less' }

Second edit:

1335309477670 { '0': 'change', '1': 'example.less' }
1335309477671 { '0': 'rename', '1': 'example.less' }
1335309477672 { '0': 'rename', '1': 'example.less' }

Later edits: nothing

[bnoordhuis]

Well, at least it's consistent with inotifywait so that's something. I've landed the patch in joyent/libuv@06ae804.

I don't understand why later edits don't show up. inotify watches the dirent (i.e. the filename), not the inode.

[trevnorris]

apparently github believes just referencing a ticket in a commit resolves it.

[danmilon]

It missed "Partially" ;)

[dougwilson]

At least with vim, this pattern occurs because the first time the file is written, a new file is created and renamed to the original. After that, the fd is kept open to that new file and changes are directly written to the file by seeking in the fd and doing writes. This is why only the first save shows up. This can be disabled by enabling the writebackup option so vim will always write a new file and then rename it for every save.

Edit: it looks like this will doesn't keep inotifywait listening to the file.

Edit 2: the option vim users need to set is backupcopy=yes so that it functions correctly.

vim with backupcopy=auto (usually the default):

# Write file
test.txt MOVE_SELF
test.txt ATTRIB
test.txt DELETE_SELF
# Write file
# Write file
# Write file

vim with backupcopy=yes:

# Write file
test.txt OPEN
test.txt ACCESS
test.txt CLOSE_NOWRITE,CLOSE
test.txt MODIFY
test.txt OPEN
test.txt MODIFY
test.txt CLOSE_WRITE,CLOSE
test.txt ATTRIB
# Write file
test.txt OPEN
test.txt ACCESS
test.txt CLOSE_NOWRITE,CLOSE
test.txt MODIFY
test.txt OPEN
test.txt MODIFY
test.txt CLOSE_WRITE,CLOSE
test.txt ATTRIB
# Write file
test.txt OPEN
test.txt ACCESS
test.txt CLOSE_NOWRITE,CLOSE
test.txt MODIFY
test.txt OPEN
test.txt MODIFY
test.txt CLOSE_WRITE,CLOSE
test.txt ATTRIB
# Write file
test.txt OPEN
test.txt ACCESS
test.txt CLOSE_NOWRITE,CLOSE
test.txt MODIFY
test.txt OPEN
test.txt MODIFY
test.txt CLOSE_WRITE,CLOSE
test.txt ATTRIB

[IonicaBizau]

I created fwatcher, a tiny wrapper for fs.watch which tries to reapend the listeners after rename event. I did it for the same reason: editing files in VIM.

var watcher = require("fwatcher");
watcher.watch('example.less', function(err, event, path) {
  console.log(+(new Date), arguments);
});

This doesn't solve the root problem, but it is a solution for the VIM users. 😄

/cc @scribu

[jasnell]

Is this still an issue?

[scribu]

I don't have an Ubuntu instance handy, but on OSX 10.10.2 and node 0.10.36 I'm still getting inconsistent events:

First edit:

1426721530261 { '0': 'rename', '1': null }

Second edit:

1426721535949 { '0': 'rename', '1': null }

Subsequent edits:

nothing

[scribu]

@IonicaBizau Thanks. Not long after opening this issue I found some other library that seemed to work better than the builtin fs.watch (I don't remember the name anymore).

[IonicaBizau]

@jasnell Still reproducing on Ubuntu 15.04.

[jasnell]

Definitely see continued inconsistency here.

[jasnell]

Marking this as a P-3 (low priority) and labeling as defer-to-convergence. This means we'll add this to the list of things to look at after we've completely merged into nodejs/node.

[pugnascotia]

Noticed this using webpack and vim. Would be great to see a fix.

[ekkis]

I'm using NPM's grunt-contrib-watch and saving a file in vim fires events twice running on OSX 10.11.2 (El Capitan). looking for a solution...