Directory Notifications
Closed this issue · 5 comments
When attempting to recursively watch directories, it would be useful to be able to receive notifications when directories are created/removed. For instance, if I am trying to watch all files contained recursively in a directory, I would need to know when directories are added in order to begin watching them as well.
I know this is supported by some existing inotify interfaces (pyinotify at least), but am not sure how easy this would be on the Windows side.
As long as you are watching the parent directory where the new directory is created, you should be getting the event. You would then just have to check each Create event to see if it is a directory, and if so, watch it.
Something like this: https://gist.github.com/howeyc/4762676
Unless I totally misunderstood your question.
Doh - sorry for the noise!
I was getting events for regular files but not directories, and I didn't find anything in the docs or in a cursory scan of the code to indicate it was detecting directory creation, so I assumed it wasn't. Instead, I had a strange case in my code that was causing directory creations to silently err, while allowing regular files to work properly.
Thanks for setting me straight.
Actually, I was trying the same thing and ran into the following problem: I start by watching for notifications in, say, /tmp. As directories are created, they are signalled so I add those to watcher. However, very quickly the system runs out of file handles.
After a quick look at the fsnotify_bsd.go code, it seems every file within a folder is opened and added to the kevent list (this on OS X) whereas all that should be needed (I think?) is a kevent monitor on the directory.
To get around the problem I ended up rolling my own stripped down solution based on code from wait_on. This works but I would much prefer to use fsnotify as it is much more versatile, though in this case it simply uses too many resources.
Here is my hacked version of example_test.go. A simple way to reproduce the problem (again on OS X) was to launch the test on /tmp, then cp -a my iphoto album with 450 or so images to /tmp.
Only adding kevent for the directory would work if all you cared about was adding/removing files from directories (ie. only modifications to directories) but you wouldn't catch when the files themselves are modified.
As for the limit, I know there's some sysctl variables on FreeBSD that can be modified, not sure if they exist for OS X as well, or even if that would solve your problem: kern.maxfilesperproc and kern.maxfiles.
I have had people mention before that it would be nicer to have more fine-grained control over what events are registered to be monitored by the OS, but I'm not really sure on a solution to allow for that.
I saw the sysctl variables but don't feel comfortable tweaking system parameters just for my app, particularly if they might affect other programs.
Using kevent is sufficient for my needs since it tells me when anything is added, modified or deleted in folder. Since it doesn't report which file, I have to scan the directory for details.
Clearly there is a trade off: to track many changes across relatively few files, your way works best. But when tracking not so many changes to many files across many directories, having the option to just monitor directory changes would be useful. It's a shame they decided not to implement this in the operating system.
Otherwise, thanks for a wonderful library - I have several other applications where it will work perfectly!