Current Error Handling does not return false on logging errors

[obokaiwele]

There are two types of errors I've run into:

1. Network Errors:
   Here I created an app to log to a remote Fluentd instance.
   private static final FluentLogger LOG = FluentLogger.getLogger("audit", "10.0.0.1", 22224);
   However, I forgot to start the remote Fluentd instance before running the app, so I got Network Errors
   java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.fluentd.logger.sender.RawSocketSender.connect(RawSocketSender.java:85) at org.fluentd.logger.sender.RawSocketSender.reconnect(RawSocketSender.java:98) at org.fluentd.logger.sender.RawSocketSender.flush(RawSocketSender.java:206) at org.fluentd.logger.sender.RawSocketSender.send(RawSocketSender.java:195) at org.fluentd.logger.sender.RawSocketSender.emit(RawSocketSender.java:157) at org.fluentd.logger.sender.RawSocketSender.emit(RawSocketSender.java:139) at org.fluentd.logger.sender.RawSocketSender.emit(RawSocketSender.java:134) at org.fluentd.logger.FluentLogger.log(FluentLogger.java:101) at org.fluentd.logger.FluentLogger.log(FluentLogger.java:86) at LoggerInfra.log(FluentdLoggerApp.java:27) at LoggerCaller.testLogger(FluentdLoggerApp.java:79) at FluentdLoggerApp.main(FluentdLoggerApp.java:112)

There error was printed to console (expected because of limited error handling #15), but the log() call returned true instead of false which hid the internal failure.
Result of logic in RawSocketSender.java#L204 which simple logs the error message to LOG.error().

2. A second error is that I ran out of buffer space. In this setup I had my app logging to a local Fluentd instance and buffering to file. The local instance then forwards the logs to a remote instance. However the remote instance was not running, so the local buffer space eventually got filled up. Again the log() call returned true and the stack trace was printed out to terminal.

[repeatedly]

How about using fluency instead of fluent-logger-java?
We are now considering replacing fluent-logger-java with fluency.

[komamitsu]

@obokaiwele

1

You can set an ErrorHandler if you want to do something other than just logging like

        FluentLogger logger = FluentLogger.getLogger(null);
        logger.setErrorHandler(new ErrorHandler() {
            @Override
            public void handleNetworkError(IOException ex)
            {
                // Notify me via email or something
            }
        });

As for the return value of log(), it returns true if it can store an event in memory since fluent-logger is going to retry to send the buffered event later.

2

I guess fluent-logger didn't notice the buffer full of the Fluentd.

in_forward/out_forward plugins of Fluentd support ACK response http://docs.fluentd.org/v0.12/articles/out_forward#requireackresponse. Without this feature, clients of in_forward plugin can't notice the failures of sending to in_forward plugin.

Unfortunately, fluent-logger doesn't support the feature and there is no plan to support it for now. But Fluency, which is yet another fluent logger, supports it https://github.com/komamitsu/fluency#enable-ack-response-mode. On the other hand, Fluency doesn't support custom error handler, but I'll implement the feature later.

[komamitsu]

Unfortunately, fluent-logger doesn't support the feature and there is no plan to support it for now. But Fluency, which is yet another fluent logger, supports it https://github.com/komamitsu/fluency#enable-ack-response-mode. On the other hand, Fluency doesn't support custom error handler, but I'll implement the feature later.

Fluency 1.4.0 now supports custom error handler and user can know and handle a send error.