Under Logstash 7.2.0, crashes reading zipped files

[yaauie]

On version 7.2.0 logstash is throwing an error when we try to open a gzip file in read mode, on version 7.1.1 I couldn't reproduce the error.

Below I'm providing the details of how to reproduce the error with docker:
On version 7.1.1 - success:

#cmd
echo -ne 'foo\nbar\n' | gzip -f > test.gz && docker run -ti --rm --name logstash -v $(pwd):/tmp/files docker.elastic.co/logstash/logstash:7.1.1 logstash -e 'input { file { mode => "read" path => "/tmp/files/test.gz"}} output {stdout {codec => "rubydebug"}}'

#output
{
       "message" => "foo",
      "@version" => "1",
    "@timestamp" => 2019-07-17T13:36:24.185Z,
          "path" => "/tmp/files/test.gz",
          "host" => "e4a683394a52"
}
{
       "message" => "bar",
      "@version" => "1",
    "@timestamp" => 2019-07-17T13:36:24.238Z,
          "path" => "/tmp/files/test.gz",
          "host" => "e4a683394a52"
}

• On version 7.2.0 - Failure:

#cmd
echo -ne 'foo\nbar\n' | gzip -f > test.gz && docker run -ti --rm --name logstash -v $(pwd):/tmp/files docker.elastic.co/logstash/logstash:7.2.0 logstash -e 'input { file { mode => "read" path => "/tmp/files/test.gz"}} output {stdout {codec => "rubydebug"}}'

#output (log)
[2019-07-17T13:40:04,837][INFO ][filewatch.observingread  ] START, creating Discoverer, Watch with file and sincedb collections
[2019-07-17T13:40:04,847][ERROR][logstash.javapipeline    ] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::File mode=>"read", path=>["/tmp/files/test.gz"], id=>"c2cd75cf15e137bd341687848d998bd336c6cce0432485d59b5246468994dd7f", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_a45f2b23-bd9d-4a04-93c5-f837cec99036", enable_metric=>true, charset=>"UTF-8">, stat_interval=>1.0, discover_interval=>15, sincedb_write_interval=>15.0, start_position=>"end", delimiter=>"\n", close_older=>3600.0, file_completed_action=>"delete", sincedb_clean_after=>1209600.0, file_chunk_size=>32768, file_chunk_count=>140737488355327, file_sort_by=>"last_modified", file_sort_direction=>"asc">
  Error: wrong number of arguments (given 1, expected 0)
  Exception: ArgumentError
  Stack: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/handlers/read_zip_file.rb:26:in `handle_specifically'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/handlers/base.rb:26:in `handle'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/processor.rb:39:in `read_zip_file'

[yaauie]

Since Logstash 7.1.1 and Logstash 7.2.0 both shipped version 4.1.0 of this plugin, something else is likely the root cause. I'm going to look into changes to the jruby we bundle or to Logstash itself that could cause this issue.

---

We are sending a message from the plugin in Ruby to an object defined in Java, with a different number of arguments than the receiver is willing to accept.

We can see from the below that the receiver is java.io.BufferedReader and that we are trying to send java.io.BufferedReader#readLine(Boolean):

java_import java.io.BufferedReader

-- lib/filewatch/read_mode/handlers/read_zip_file.rb:6@v4.1.10

        buffered = BufferedReader.new(decoder)
        while (line = buffered.readLine(false))

-- lib/filewatch/read_mode/handlers/read_zip_file.rb:25-26@v4.1.10

But the docs for java.io.BufferedReader indicates that while it responds to readLine, it does not have an overloaded method that accepts any arguments.

I am not sure how this ever worked.

[yaauie]

Logstash 7.2.0 and Logstash 7.1.1 both include the same JRuby 9.2.7.0 😕

[TheVastyDeep]

I think it is the version of Java that determines whether this blows up or not. It does not happen with

jruby 9.2.7.0 (2.5.3)
OpenJDK 64-Bit Server VM 25.201-b09 on 1.8.0_201-b09

[nqkdev]

The same problem for logstash docker image 7.2.0 and 7.3.0. Is it the incompatibility with java11 that causes the issue?

[guyboertje]

I am not sure how this ever worked.

I am not certain either. Perhaps under jdk 11, jruby has a more strict method signature choice?

[eladtamary]

Tried it with Logstash 7.4.2 and it's working.

[andsel]

I've done some local test with file-input in various Logstash version running on JDK8 and JDK11, the results are:

	JDK 8	JDK 11
Logstash 7.1.1	OK	OK
Logstash 7.2.0	Fail	Fail
Logstash 7.5.1	OK	OK

@yaauie we can close as not fix it because seems related to an old version?

[roaksoax]

Seems this issue was addressed by this report was kept open. I'm gonna go head and close it.