dwbutler/logstash-logger

Buffer not working with multi_logger

Closed this issue · 4 comments

When using multi_logger with udp and stdout nothing is pushed into logstash
The buffer_max_items is not recognize and I get
[LogStashLogger::Device::UDP] Buffer Full - {:pending=>50, :outgoing=>0}

    config.log_level = :debug
    config.autoflush_log = true
    config.logstash.error_logger = Logger.new(STDOUT)
    config.logstash.buffer_max_items = 1000
    config.logstash.buffer_max_interval = 1
    config.logstash.drop_messages_on_flush_error = false
    config.logstash.drop_messages_on_full_buffer = true
    config.logstash.type = :multi_logger
    config.logstash.outputs = [
      {
        type: :stdout,
        formatter: ::Logger::Formatter
      },
      {
        type: :udp,
        port: 5228,
        host: '11.11.3.253',
        formatter: :json_lines,
      }
    ]

I checked the source for MultiLogger, and in fact none of the options passed in to the LogStashLogger constructor are passed down into each constructed logger of the MultiLogger. This certainly seems like surprising behavior that ought to be fixed.

This could also explain why nothing seems to be pushed into logstash. You specified autoflush_log true, but this option isn't passed down into the UDP logger. So it defaults to flushing every 5 seconds instead.

This issue should now be resolved in 0.21.0.

In case anyone finds this by searching on the error - I get this occasionally if sync: true is missing from a MultiLogger config item pointed to a UDP port.

i got this message
[2020-01-17T14:22:37.511422 #41302] WARN -- : [LogStashLogger::Device::UDP] Buffer Full - {:pending=>50, :outgoing=>0}

@srogers aweasome thanks!!
I did digging this problem for a few days
i added sync:true, then buffer full message is gone and ELK get nomal log

config.lograge.logger = LogStashLogger.new(type: :udp, host: 'localhost', port: 5000, sync: true)