Missing Ruby class handling for full class name=java.util.HashMap

Question

Missing Ruby class handling for full class name=java.util.HashMap

jpcarey opened this issue 7 years ago · 20 comments

Version: 5.5.0 & 5.4.2
Config File:

input { stdin {} }
#stdin: 8.8.8.8,8080

filter {
  csv { columns => [ "pan_destination_ip", "pan_destination_port" ]}
  mutate {
    add_field => {
      "[destination][ip]" => "%{pan_destination_ip}"
      "[destination][port]" => "%{pan_destination_port}"
    }
  }
  geoip {
    source => "[destination][ip]"
    target => "destination"
  }
}

output { stdout { codec => rubydebug } }

Sample Data: 8.8.8.8,8080

[2017-07-11T17:01:18,534][ERROR][logstash.pipeline        ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=>"Missing Ruby class handling for full class name=java.util.HashMap, simple name=HashMap", "backtrace"=>["org.logstash.Javafier.deep(org/logstash/Javafier.java:29)", "org.logstash.Event.getField(org/logstash/Event.java:147)", "org.logstash.filters.GeoIPFilter.applyGeoData(org/logstash/filters/GeoIPFilter.java:156)", "org.logstash.filters.GeoIPFilter.handleEvent(org/logstash/filters/GeoIPFilter.java:148)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:497)", "RUBY.filter(/Users/jared/builds/logstash/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.2.1-java/lib/logstash/filters/geoip.rb:122)", "RUBY.do_filter(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:145)", "RUBY.multi_filter(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:164)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.multi_filter(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:161)", "RUBY.multi_filter(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/filter_delegator.rb:43)", "RUBY.filter_func((eval):96)", "LogStash::Pipeline.filter_batch(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:383)", "LogStash::Pipeline.filter_batch(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:383)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:238)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:238)", "org.jruby.RubyHash.each(org/jruby/RubyHash.java:1342)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:237)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:237)", "LogStash::Pipeline.filter_batch(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:382)", "LogStash::Pipeline.filter_batch(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:382)", "RUBY.worker_loop(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:363)", "RUBY.start_workers(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:330)", "java.lang.Thread.run(java/lang/Thread.java:745)"]}
Exception in thread "[main]>worker1" java.lang.IllegalArgumentException: Missing Ruby class handling for full class name=java.util.HashMap, simple name=HashMap
	at org.logstash.Javafier.deep(org/logstash/Javafier.java:29)
	at org.logstash.Event.getField(org/logstash/Event.java:147)
	at org.logstash.filters.GeoIPFilter.applyGeoData(org/logstash/filters/GeoIPFilter.java:156)
	at org.logstash.filters.GeoIPFilter.handleEvent(org/logstash/filters/GeoIPFilter.java:148)
	at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:497)
	at RUBY.filter(/Users/jared/builds/logstash/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.2.1-java/lib/logstash/filters/geoip.rb:122)
	at RUBY.do_filter(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:145)
	at RUBY.multi_filter(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:164)
	at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
	at RUBY.multi_filter(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:161)
	at RUBY.multi_filter(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/filter_delegator.rb:43)
	at RUBY.filter_func((eval):96)
	at LogStash::Pipeline.filter_batch(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:383)
	at LogStash::Pipeline.filter_batch(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:383)
	at org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)
	at LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:238)
	at LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:238)
	at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1342)
	at LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:237)
	at LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:237)
	at LogStash::Pipeline.filter_batch(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:382)
	at LogStash::Pipeline.filter_batch(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:382)
	at RUBY.worker_loop(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:363)
	at RUBY.start_workers(/Users/jared/builds/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:330)
	at java.lang.Thread.run(java/lang/Thread.java:745)

Answer 1 · 2017-07-13T15:55:06.000Z

here's a simpler test:

input { generator { count => 1 add_field => { client => "8.8.8.8" } } }
filter{
  geoip {
    source => "client"
    target => "client"
  }
}

Results in:

/tmp/logstash-5.5.0 % bin/logstash -f ../logstash-5.5.0/cfg2
[2017-07-13T16:53:16,195][INFO ][logstash.filters.geoip   ] Using geoip database {:path=>"/tmp/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.2.1-java/vendor/GeoLite2-City.mmdb"}
[2017-07-13T16:53:16,227][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-07-13T16:53:16,232][INFO ][logstash.pipeline        ] Pipeline main started
[2017-07-13T16:53:16,342][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2017-07-13T16:53:16,472][ERROR][logstash.pipeline        ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=>"expecting List or Map, found class org.logstash.bivalues.StringBiValue", "backtrace"=>["org.logstash.Accessors.newCollectionException(org/logstash/Accessors.java:195)", "org.logstash.Accessors.store(org/logstash/Accessors.java:182)", "org.logstash.Accessors.set(org/logstash/Accessors.java:26)", "org.logstash.Event.setField(org/logstash/Event.java:170)", "org.logstash.filters.GeoIPFilter.applyGeoData(org/logstash/filters/GeoIPFilter.java:166)", "org.logstash.filters.GeoIPFilter.handleEvent(org/logstash/filters/GeoIPFilter.java:148)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "RUBY.filter(/tmp/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.2.1-java/lib/logstash/filters/geoip.rb:122)", "RUBY.do_filter(/tmp/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:145)", "RUBY.multi_filter(/tmp/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:164)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.multi_filter(/tmp/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:161)", "RUBY.multi_filter(/tmp/logstash-5.5.0/logstash-core/lib/logstash/filter_delegator.rb:43)", "RUBY.filter_func((eval):38)", "RUBY.filter_batch(/tmp/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:383)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "RUBY.each(/tmp/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:238)", "org.jruby.RubyHash.each(org/jruby/RubyHash.java:1342)", "RUBY.each(/tmp/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:237)", "RUBY.filter_batch(/tmp/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:382)", "RUBY.worker_loop(/tmp/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:363)", "RUBY.start_workers(/tmp/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:330)", "java.lang.Thread.run(java/lang/Thread.java:745)"]}
Exception in thread "[main]>worker2" java.lang.ClassCastException: expecting List or Map, found class org.logstash.bivalues.StringBiValue
	at org.logstash.Accessors.newCollectionException(org/logstash/Accessors.java:195)
	at org.logstash.Accessors.store(org/logstash/Accessors.java:182)
	at org.logstash.Accessors.set(org/logstash/Accessors.java:26)
	at org.logstash.Event.setField(org/logstash/Event.java:170)
	at org.logstash.filters.GeoIPFilter.applyGeoData(org/logstash/filters/GeoIPFilter.java:166)
	at org.logstash.filters.GeoIPFilter.handleEvent(org/logstash/filters/GeoIPFilter.java:148)
	at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
	at RUBY.filter(/tmp/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.2.1-java/lib/logstash/filters/geoip.rb:122)
	at RUBY.do_filter(/tmp/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:145)
	at RUBY.multi_filter(/tmp/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:164)
	at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
	at RUBY.multi_filter(/tmp/logstash-5.5.0/logstash-core/lib/logstash/filters/base.rb:161)
	at RUBY.multi_filter(/tmp/logstash-5.5.0/logstash-core/lib/logstash/filter_delegator.rb:43)
	at RUBY.filter_func((eval):38)
	at RUBY.filter_batch(/tmp/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:383)
	at org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)
	at RUBY.each(/tmp/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:238)
	at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1342)
	at RUBY.each(/tmp/logstash-5.5.0/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:237)
	at RUBY.filter_batch(/tmp/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:382)
	at RUBY.worker_loop(/tmp/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:363)
	at RUBY.start_workers(/tmp/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:330)
	at java.lang.Thread.run(java/lang/Thread.java:745)

Answer 2 · 2017-07-24T14:15:51.000Z

I have reproduced the above error in logstash 5.4.3 and 5.5.0 (and I can confirm that I am using the default elasticsearch mapping for geoip).

Error from Logstash 5.5.0:

[2017-07-24T14:10:19,857][ERROR][logstash.pipeline ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=>"Missing Ruby class handling for full class name=java.util.HashMap, simple name=HashMap", "backtrace"=>["org.logstash.Javafier.deep(org/logstash/Javafier.java:29)", "org.logstash.Event.getField(org/logstash/Event.java:147)", "org.logstash.filters.GeoIPFilter.applyGeoData(org/logstash/filters/GeoIPFilter.java:156)", "org.logstash.filters.GeoIPFilter.handleEvent(org/logstash/filters/GeoIPFilter.java:148)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "RUBY.filter(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.2.1-java/lib/logstash/filters/geoip.rb:122)", "RUBY.do_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145)", "RUBY.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161)", "RUBY.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:43)", "RUBY.initialize((eval):55082)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.initialize((eval):55078)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "RUBY.filter_func((eval):10462)", "LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:383)", "LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:383)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:238)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:238)", "org.jruby.RubyHash.each(org/jruby/RubyHash.java:1342)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:237)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:237)", "LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:382)", "LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:382)", "RUBY.worker_loop(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:363)", "RUBY.start_workers(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:330)", "java.lang.Thread.run(java/lang/Thread.java:748)"]}

Answer 3 · 2017-07-25T14:32:49.000Z

This error shows itself when the target field exists and is not a k/v map. I have fixed this now.

A simple Java test recreated the error. PR to follow.

Answer 4 · 2017-07-25T14:37:31.000Z

Thanks @guyboertje,

My "target" has nested property fields. In case that helps.

Answer 5 · 2017-07-25T15:29:06.000Z

@zaakiy
How did your target become a k/v map? I mean what inputs/filters or add_field did you use to "set" the target in such a way?

Answer 6 · 2017-07-25T16:19:40.000Z

@jsvd
BTW, your failure is not exactly the same as the previous ones.
Yours fails at: org.logstash.Event.setField(org/logstash/Event.java:170)
Others at: org.logstash.Event.getField(org/logstash/Event.java:147)

Answer 7 · 2017-07-25T21:27:58.000Z

@guyboertje,

(Edited)

PaloAltoTraffic is created by CSV filter:

                csv {
                    add_tag => [ "1cdeaa86" ]
                    source => "message"
                    columns => [ "FUTURE_USE1","ReceiveTime","SerialNumber","Type","Subtype","FUTURE_USE2","GeneratedTime","SourceIP","DestinationIP","NATSourceIP","NATDestinationIP","RuleName","SourceUser","DestinationUser","Application","VirtualSystem","SourceZone","DestinationZone","IngressInterface","EgressInterface","LogForwardingProfile","FUTURE_USE3","SessionID","RepeatCount","SourcePort","DestinationPort","NATSourcePort","NATDestinationPort","Flags","Protocol","Action","Bytes","BytesSent","BytesReceived","Packets","StartTime","ElapsedTime","Category","FUTURE_USE4","SequenceNumber","ActionFlags","SourceLocation","DestinationLocation","FUTURE_USE5","PacketsSent","PacketsReceived","SessionEndReason","DeviceGroupHierarchyLevel1","DeviceGroupHierarchyLevel2","DeviceGroupHierarchyLevel3","DeviceGroupHierarchyLevel4","VirtualSystemName","DeviceName","ActionSource" ]
                    target => "PaloAltoTraffic"
                }

Then the field is copied...

mutate {
                    add_field => { "[Source][IP]" => "%{[PaloAltoTraffic][SourceIP]}" }
}

Then geoip

geoip {
            source => "[Source][IP]"
            target => "Source"
}

Answer 8 · 2017-07-26T07:44:59.000Z

@zaakiy
Ahh, OK, its the same mechanism as the original post - using add_field.

Answer 9 · 2017-07-26T07:46:05.000Z

There are two bugs at work here. One (identified and fixed) in the geoip filter and the other is in Logstash core (identified).

Answer 10 · 2017-07-26T08:48:21.000Z

Thanks @guyboertje. Can you tell me what version of geoip filter it was fixed in?

If you could kindly link to your PR when you are ready, I can test it for you if instructions can be provided (have never tested a Logstash patch before).

Answer 11 · 2017-07-26T12:19:48.000Z

@zaakiy - Unfortunately, the bug you are experiencing is not in the geoip filter, its in logstash core.

Answer 12 · 2017-07-26T12:48:55.000Z

@zaakiy
I have a work around for this problem. It involves using the json filter to create the correct kind of object via add_field.

  csv {
      add_tag => [ "1cdeaa86" ]
      source => "message"
      columns => [ "FUTURE_USE1","ReceiveTime","SerialNumber","Type","Subtype","FUTURE_USE2","GeneratedTime","SourceIP","DestinationIP","NATSourceIP","NATDestinationIP","RuleName","SourceUser","DestinationUser","Application","VirtualSystem","SourceZone","DestinationZone","IngressInterface","EgressInterface","LogForwardingProfile","FUTURE_USE3","SessionID","RepeatCount","SourcePort","DestinationPort","NATSourcePort","NATDestinationPort","Flags","Protocol","Action","Bytes","BytesSent","BytesReceived","Packets","StartTime","ElapsedTime","Category","FUTURE_USE4","SequenceNumber","ActionFlags","SourceLocation","DestinationLocation","FUTURE_USE5","PacketsSent","PacketsReceived","SessionEndReason","DeviceGroupHierarchyLevel1","DeviceGroupHierarchyLevel2","DeviceGroupHierarchyLevel3","DeviceGroupHierarchyLevel4","VirtualSystemName","DeviceName","ActionSource" ]
      target => "PaloAltoTraffic"
      add_field => {
        "Source" => '{"IP":"%{[PaloAltoTraffic][SourceIP]}"}'
      }
  }

  json {
    source => "Source"
    target => "Source"
  }

  geoip {
    source => "[Source][IP]"
    target => "Source"
  }

NOTE: the add_field in the csv filter, it creates a dynamic JSON string with a value from the Event. This is OK because the add_field is done after the csv adds the field/values

Answer 13 · 2017-07-26T13:17:46.000Z

@guyboertje, I can't thank you enough for that workaround!

Answer 14 · 2017-08-05T13:42:56.000Z

@guyboertje, unfortunately the workaround does not work. :-(

[2017-08-05T23:40:29,940][WARN ][logstash.filters.json    ] Error parsing json {:source=>"PaloAltoTraffic", :raw=>{"RepeatCount"=>"1", "IngressInterface"=>"ae2.19", "DestinationPort"=>"443", "GeneratedTime"=>"2017/08/05 22:29:16", "Bytes"=>"4832", 

SNIPPED

}, :exception=>java.lang.ClassCastException: org.jruby.RubyHash cannot be cast to org.jruby.RubyIO}

Update: added more detail.

Answer 15 · 2017-08-07T10:49:40.000Z

@zaakiy
Different error. The JSON filter is saying that it can't convert a Hash structure to an IO structure. The JSON filter wants to read text from a file (IO) or a string (StringIO).
Look carefully at your add_field directive. Mine was:

      add_field => {
        "Source" => '{"IP":"%{[PaloAltoTraffic][SourceIP]}"}'
      }

You should see that the field being added is a string called Source and the value of that field is a string in single quotes '{"IP":"%{[PaloAltoTraffic][SourceIP]}"}'. This string is the JSON source that the JSON filter will decode the Hash structure and replace the Source fields value with.

Correct this or post your config (I know my suggestion works because I tested it locally).

Answer 16 · 2017-08-07T10:59:31.000Z

If you need a online JSON string generator, try http://objgen.com/json?demo=true
type in the key = value and it will generate the JSON string representation. Copy and paste that into the single quote string - remember to remove the newlines.

Answer 17 · 2017-08-07T12:55:59.000Z

Hi @guyboertje,

Thank you for the additional explanation. I had missed the "IP" part of the JSON by mistake.

I've tested that this works on Logstash 5.5.1 and can confirm that your workaround works like a charm!

Many thanks again.

Answer 18 · 2017-08-07T13:16:14.000Z

Future readers: The above workaround is only needed if you can't upgrade to LS 5.6 or 6.0.

However as a technique it works great if you need to create a deep Hash/Array structure in your event ahead of other filters populating said Hash/Array structure. I plan to release a "Shaper" filter to do this without needing a JSON parse step - i.e. you need to shape your event in some way. I see this Shaper filter being the first filter in the config.

Answer 19 · 2018-02-26T14:38:16.000Z

Hi,

I ran into this problem upgrading from 5.4.1 to 5.4.3 and had to downgrade to keep things working. My plan going forward is to upgrade to 5.6 and on to 6.x when everything is working. Just to see if I've understood this correctly: this problem arises when the target for GeoIp-filter points to a field that exists but is not of the type expected by the plugin? I was a bit confused by the example workaround @guyboertje recommended (thanks!) above since it was manipulating the "Source"-attribute, but maybe that confusion was just because of the naming of the field.

I have a number of GeoIp-filters in my pipeline so I wanted to make sure I've understood this issue before I try to change it all... So, is it correct to assume that if I make sure all fields used as target are created the right way, things should work? Also, what is the "right" way?

Thanks!

Answer 20 · 2018-05-04T09:11:39.000Z

Closing.
Fixed in newer versions of plugin and LS.