Fix polling behavior
samoht opened this issue · 0 comments
samoht commented
Currently, DataKit rely on filesystem polling + scanning to detect file system changes. This is slow and CPU intensive. We should git rid of that.
To do so, I have created https://github.com/samoht/irmin-watcher which abstract the Irmin watch API over various backend (inotify/fsevents). Unfortunately, this is not ready to be deployed yet, as:
- it is unreliable, it crashes on OSX due to dsheets/ocaml-osx-cf#8
- it seems to crash on Linux too (need to investigate more and have a repro)
- Irmin until recently didn't really like watch callback to raise exception, but this is now fixed in mirage/irmin#375 (need a new release of Irmin to happen)
- it is slow, as although it relies on FS events, it still scans the entire tree on every event. This is clearly not very optimal :-)