File changes not detected in a certain condition

[kwastek]

I try to detect changes in a log file generated by picocom:
picocom -b 115200 --imap lfcrlf /dev/ttyACM0 >> testfile.log
In the second console I monitor the file with tail:
tail -f testfile.log
In the third console I try do detect the changes with entr:
echo testfile.log | entr -sn "echo change"

tail shows all file changes as they happen. So in principle it's possible to monitor the file changes. However, entr doesn't detect the changes. entr shows there was a change only after I kill picocom.

EDIT:
Python utility when-changed (pip install when-changed) is also able to detect the changes:
when-changed testfile.log echo change
So I'm sure entr should be also able to detect changes if it really tried ;)

[eradman]

This is as intended:

On Linux entr waits until the file closed (IN_CLOSE_WRITE) so that it does not launch utilities before a file is completely written. (Downloading a file for example.)

Unfortunately kqueue does not prove this insight, so on BSD entr may run the utility on a file that is partially written.

[kwastek]

That's a shame :(

I don't mind entr reacting to each small file change. In my script I use sleep at the beginning, before I process the modified file. That way I give time for the serial transmission to finish. I ended up using the when-changed utility. But I think it would be nice for entr to have this functionality as well.

I understand your reasoning, but I think it would be great to have a switch for this behavior.

[eradman]

entr is a test runner, so the primary objective is to avoid spurious changes by collapsing events so that

1. a build is not triggered prematurely
2. a build is only invoked only once

What your use case? If you're looking to respond to all events, perhaps inotifywait is the right tool

inotifywait -q -e modify testfile.log --format "%w%f"

[kwastek]

Thank you very much. It is even able to react to changes in all files in a folder:

inotifywait -q -e modify SerialLogs/ --format "%w%f"

My use case is that I capture logs from a serial console of an embedded devices, and want to send an email when any of the devices resets. So I want some tool that will notice a change in the log file, so I'm able to grep it for more details and then take further actions.

I think for now I'll stay with the when-changed utility since it works the way I need it and also is able to directly execute a command like entr does. But if it turns out I need more control over what is happening, then I'll go with the inotifywait route.

Thank you for taking the time to help me!

[eradman]

No problem! I see why you would want to do that.

If you'd like to build entr from source it can be easily patched. Something like this:

diff --git a/missing/kqueue_inotify.c b/missing/kqueue_inotify.c
index 7891dda..eaa5d2b 100644
--- a/missing/kqueue_inotify.c
+++ b/missing/kqueue_inotify.c
@@ -81,7 +81,7 @@ fs_sysctl(const int name) {

 #define EVENT_SIZE (sizeof (struct inotify_event))
 #define EVENT_BUF_LEN (32 * (EVENT_SIZE + 16))
-#define IN_ALL IN_CLOSE_WRITE|IN_DELETE_SELF|IN_MOVE_SELF|IN_MOVE|IN_ATTRIB|IN_CREATE|IN_DELETE
+#define IN_ALL IN_MODIFY|IN_DELETE_SELF|IN_MOVE_SELF|IN_MOVE|IN_ATTRIB|IN_CREATE|IN_DELETE

 /*
  * inotify and kqueue ids both have the type `int`
@@ -193,7 +193,7 @@ kevent(int kq, const struct kevent *changelist, int nchanges, struct
                                /* convert iev->mask; to comparable kqueue flags */
                                fflags = 0;
                                if (iev->mask & IN_DELETE_SELF) fflags |= NOTE_DELETE;
-                               if (iev->mask & IN_CLOSE_WRITE) fflags |= NOTE_WRITE;
+                               if (iev->mask & IN_MODIFY)      fflags |= NOTE_WRITE;
                                if (iev->mask & IN_CREATE)      fflags |= NOTE_WRITE;
                                if (iev->mask & IN_DELETE)      fflags |= NOTE_WRITE;
                                if (iev->mask & IN_MOVE_SELF)   fflags |= NOTE_RENAME;