Logstash parsing error

Question

Logstash parsing error

ngms17 opened this issue 4 years ago · 9 comments

Answer 1 · 2020-12-01T16:31:10.000Z

Can you please edit your issue to provide the actual log text instead of a screenshot? Also, please provide more details of your environment. Version of Suricata? Version of Elastic Stack? etc.

Answer 2 · 2020-12-01T16:32:38.000Z

[2020-12-01T15:40:07,925][WARN ][logstash.outputs.elasticsearch][synlite_suricata][7f0f636925cafdc45ccbf6445a1562dacede6781ba4cf6f1b34e30bf21e877ba] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"suricata-1.1.0-2020.12.01", :routing=>nil, :_type=>"_doc"}, #LogStash::Event:0x24ead3be], :response=>{"index"=>{"_index"=>"suricata-1.1.0-2020.12.01", "_type"=>"_doc", "_id"=>"PuD2HnYBONJZyV17Akbk", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [dns.grouped.SOA] of type [keyword] in document with id 'PuD2HnYBONJZyV17Akbk'. Preview of field's value: '{rname=hostmaster.inesctec.pt, serial=2020112712, expire=1209600, refresh=7200, mname=ns.inesctec.pt, minimum=21600, retry=3600}'", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:1874"}}}}}

Answer 3 · 2020-12-01T16:39:42.000Z

You will probably need to modify your suricata configuration to send the old style of DNS log. This is done in suricata.yml under the eve-log -> types section. You need to use version 1. For example...

        - dns:
            enabled: yes
            version: 1

Answer 4 · 2020-12-01T16:49:05.000Z

Unfortunatly it didn´t resolved the problem

Answer 5 · 2020-12-01T16:50:40.000Z

You will have to delete any indices that were already created. Also... what versions of Suricata and the Elastic Stack are you using?

Answer 6 · 2020-12-01T16:51:30.000Z

All of the recent ones

Answer 7 · 2020-12-01T16:53:24.000Z

That could be part of the problem. This solution was created using Elastic Stack 7.1.x. It hasn't been test with any of the latest versions.

Answer 8 · 2020-12-01T16:55:51.000Z

I will try that solution. If it does not work, i will have to downgrade. What version of suricata are you using?

Are you thinking of upgrading to newer versions?

Answer 9 · 2020-12-01T16:59:01.000Z

None at the moment actually. I think 4.x was current at the time this was created. I would have to spend some time to fully upgrade everything to support the latest releases. However, I won't realistically, be able to get to that until after the new year.