java.net.SocketException: Broken pipe

[felixzh2020]

2017-12-27 19:46:39,120 ERROR [org.fluentd.logger.sender.RawSocketSender] - <org.fluentd.logger.sender.RawSocketSender>

java.net.SocketException: Broken pipe

    at java.net.SocketOutputStream.socketWrite0(Native Method)

    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)

    at java.net.SocketOutputStream.write(SocketOutputStream.java:159)

    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)

    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)

    at org.fluentd.logger.sender.RawSocketSender.flush(RawSocketSender.java:189)

    at org.fluentd.logger.sender.RawSocketSender.send(RawSocketSender.java:177)

    at org.fluentd.logger.sender.RawSocketSender.emit(RawSocketSender.java:147)

    at org.fluentd.logger.sender.RawSocketSender.emit(RawSocketSender.java:129)

    at org.fluentd.logger.sender.RawSocketSender.emit(RawSocketSender.java:124)

    at org.fluentd.logger.FluentLogger.log(FluentLogger.java:101)

    at org.fluentd.logger.FluentLogger.log(FluentLogger.java:86)

    at com.hisense.hitv.api.shopping.threadpool.log.BehaviorLogSendTask.sendExtraLog(BehaviorLogSendTask.java:91)

    at com.hisense.hitv.api.shopping.threadpool.log.BehaviorLogSendTask.run(BehaviorLogSendTask.java:45)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

    at java.lang.Thread.run(Thread.java:745)

[nyilmaz]

I think this happens when fluentd somehow saturated.

[komamitsu]

If transporting events to Fluentd is a bottleneck, I recommend https://github.com/komamitsu/fluency. The logger uses PackedForward format to send a lot of events to Fluentd so that the number of sending data is significantly reduced.

[felixzh2020]

@komamitsu
ok, thanks. I will try it.

[komamitsu]

I'm closing this issue. Feel free to reopen it if you want.