logstash-plugins/logstash-input-sqlite

DB field contains an object, causing serialization issues

Opened this issue · 1 comments

logstash 2.1.0

When reading from an sqlite database the plugin seems to be generating an Event with a ruby object in the db field.

"db" => #<Sequel::JDBC::Database: "jdbc:sqlite:/tmp/myfile.sqlite">

This is preventing logstash from using this event and serializing it into json for output.

LogStash::Json::GeneratorError: Cannot find Serializer for class: org.jruby.RubyObject
     jruby_dump at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.0-  java/lib/logstash/json.rb:53
        to_json at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.0-java/lib/logstash/event.rb:158
        receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-pipe-2.0.2/lib/logstash/outputs/pipe.rb:42
         handle at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.0-java/lib/logstash/outputs/base.rb:81
    output_func at (eval):27
   outputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.0-java/lib/logstash/pipeline.rb:276
  start_outputs at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.0-java/lib/logstash/pipeline.rb:193

I was able to fix this issue by executing a mutate { remove_field => ['db'] } but it would be handy to have the database location in the event.

My guess is the longterm fix is to change the following line 160:

event = LogStash::Event.new("host" => @host, "db" => @db)

I'm not familiar with the @db object but my guess is that it should contain a method to output the database file path as a string instead of the @db object itself.

@mshirley Agreed. This is a bug. Thank you for your excellent details and code walking - should be an easy fix.