Logs are discarded when stdout is a pipe

Question

Logs are discarded when stdout is a pipe

Closed this issue 8 years ago · 7 comments

I'm using nosh (a daemontools derivative) as a process supervisor and stdout is piped to a logging process. This causes all logs to be discarded.

As a simple test I simply piped my program through cat and logs are, again, discarded. Being a pedant I even double checked with strace! When stdout is a tty I see: write(1, "...") ... but when stdout is a pipe nothing is written to file descriptor 1.

I can understand interactive programs discarding output when stdout is a pipe but surely not a logging library!

Answer 1 · 2017-02-07T10:53:04.000Z

Could you provide me with a minimal example to reproduce this problem?

Answer 2 · 2017-02-19T08:17:04.000Z

Yeah, I will do. I'm still trying to work out what's actually going on; sometimes it works and others not. I'll try and get an example together soon.

Answer 3 · 2017-02-20T04:39:37.000Z

Ok, I created a repo and have written up what I think is going on in the README.

In short I don't think this is a bug but an interaction with stdout buffering. I'm not sure what the best answer is but it might be worth at least documenting that this can happen.

I was thinking that the library could set stdout and stderr's buffer options to LineBuffering but that goes against the grain of what batching is doing. Maybe it should be set when not using the batching (I haven't tested whether this shows the same behaviour).

Answer 4 · 2017-02-20T15:31:49.000Z

Richard Heycock <notifications@github.com> writes:

Ok, I created a [repo](github.com/filterfish/logging-effect-test) and have written up what I think is going on in the [README](github.com/filterfish/logging-effect-test/blob/master/README.md). In short I don't think this is a bug but an interaction with stdout buffering. I'm not sure what the best answer is but it might be worth at least documenting that this can happen. I was thinking that the library could set stdout and stderr's buffer options to `LineBuffering` but that goes against the grain of what batching is doing. Maybe it should be set when not using the batching (I haven't tested whether this shows the same behaviour).

I don't think it should change how buffering happens, but it sounds like it just needs to explicitly flush the handle when writing. I'll add that in!

Answer 5 · 2017-02-20T15:48:42.000Z

@filterfish I have pushed what I think is a fix for this problem to the master branch - could you try that and see if it does indeed fix the problem?

Answer 6 · 2017-02-26T23:33:36.000Z

Works like a charm! Thanks for fixing it.

Answer 7 · 2017-02-27T12:55:02.000Z

Wonderful, I've now released that as 1.2.0 on Hackage. It's a major bump as this is an observable change.