Error: Unexpected end of ZLIB input stream

[KIVagant]

• Logstash Version: 5.6.8
• Plugin Version: logstash-input-s3 3.2.0
• Operating System: Ubuntu 14.04.5 LTS
• Config File:

input {
  s3 {
      add_field => ["type", "my_s3_log"]
      sincedb_path => "/etc/logstash/*****"

      access_key_id => "*****"
      secret_access_key => "******"
      bucket => "incoming-logs"
      prefix => "******/"
      backup_to_bucket => "incoming-logs-processed"
      backup_add_prefix => "******/"
      delete => true
      codec => multiline {
        patterns_dir => "/etc/logstash/patterns"
        pattern => "^(%{TIMESTAMP_ISO8601} %{INT} %{ARCHIVARIUS_LOGLEVEL})|(%{KERNEL_TIME})"
        negate => true
        what => "previous"
        max_bytes => "10MiB"
        max_lines => 5000
      }
    }
}

/var/log/logstash/logstash-plain.log:

[2018-03-07T22:31:10,969][ERROR][logstash.pipeline        ] A plugin had an unrecoverable error. Will restart this plugin.
  Plugin: <LogStash::Inputs::S3 add_field=>{"type"=>"my_s3_log"}, sincedb_path=>"/etc/logstash/.****", access_key_id=>"****", secret_access_key=>"****", bucket=>"incoming-logs", prefix=>"****", backup_to_bucket=>"incoming-logs-processed", backup_add_prefix=>"****/", delete=>true, codec=><LogStash::Codecs::Multiline patterns_dir=>["/etc/logstash/patterns"], pattern=>"^(%{TIMESTAMP_ISO8601} %{INT} %{ARCHIVARIUS_LOGLEVEL})|(%{KERNEL_TIME})", negate=>true, what=>"previous", max_bytes=>10485760, max_lines=>5000, id=>"9b44922b462112c56424c412ea022764297633e0-9", enable_metric=>true, charset=>"UTF-8", multiline_tag=>"multiline">, id=>"9b44922b462112c56424c412ea022764297633e0-10", enable_metric=>true, region=>"us-east-1", interval=>60, temporary_directory=>"/tmp/logstash">
  Error: Unexpected end of ZLIB input stream

  Exception: Java::JavaIo::EOFException
  Stack: java.util.zip.InflaterInputStream.fill(java/util/zip/InflaterInputStream.java:240)
java.util.zip.InflaterInputStream.read(java/util/zip/InflaterInputStream.java:158)
java.util.zip.GZIPInputStream.read(java/util/zip/GZIPInputStream.java:117)
sun.nio.cs.StreamDecoder.readBytes(sun/nio/cs/StreamDecoder.java:284)
sun.nio.cs.StreamDecoder.implRead(sun/nio/cs/StreamDecoder.java:326)
sun.nio.cs.StreamDecoder.read(sun/nio/cs/StreamDecoder.java:178)
java.io.InputStreamReader.read(java/io/InputStreamReader.java:184)
java.io.BufferedReader.fill(java/io/BufferedReader.java:161)
java.io.BufferedReader.readLine(java/io/BufferedReader.java:324)
java.io.BufferedReader.readLine(java/io/BufferedReader.java:389)
java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
RUBY.read_gzip_file(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-3.2.0/lib/logstash/inputs/s3.rb:269)
RUBY.read_file(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-3.2.0/lib/logstash/inputs/s3.rb:250)
RUBY.process_local_log(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-3.2.0/lib/logstash/inputs/s3.rb:178)
RUBY.process_log(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-3.2.0/lib/logstash/inputs/s3.rb:349)
RUBY.process_files(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-3.2.0/lib/logstash/inputs/s3.rb:152)
org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
RUBY.process_files(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-3.2.0/lib/logstash/inputs/s3.rb:147)
RUBY.run(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-3.2.0/lib/logstash/inputs/s3.rb:103)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)
RUBY.interval(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.23/lib/stud/interval.rb:20)
RUBY.run(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-3.2.0/lib/logstash/inputs/s3.rb:102)
RUBY.inputworker(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:470)
RUBY.start_input(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:463)
java.lang.Thread.run(java/lang/Thread.java:748)

[KIVagant]

I think, the problem is that the archive in the bucket is corrupted. I believe the plugin should not crash in this situation and just log some error message.

[KIVagant]

Looks like @shuwada has tried to fix this: #70 for the next issue #31

The PR is pretty old. Current code looks like:

private
  def read_gzip_file(filename, block)
    begin
      Zlib::GzipReader.open(filename) do |decoder|
        decoder.each_line { |line| block.call(line) }
      end
    rescue Zlib::Error, Zlib::GzipFile::Error => e
      @logger.error("Gzip codec: We cannot uncompress the gzip file", :filename => filename)
      raise e
    end
  end

As I see, only some of Zlib exceptions can be caught here and they still be raised after. In my case it's an another type of exception: Java::JavaIo::EOFException. I feel like any exceptions should be caught here and logged as error without raising them.

[KIVagant]

Since the PR was merged, I'm closing the ticket.